5
$\begingroup$

I have an ordinal variable, overall_tumor_grade, that can take on values of 1, 2, 3, or X if the measurement is indeterminable. There are some NAs that I want to impute using the mice package in R, but I know that the missing values cannot be X because their tumor sizes are greater than 0. I want to impute overall_tumor_grade but force mice to only choose from 1, 2, or 3.

Here is sample code for you to use:

df=data.frame(age=c(24,37,58,65,70,84),
    overall_tumor_grade=c(1,1,2,3,'X',NA),
    tumor_size=c(1.5,2.0,4.2,5.6,0,0.1))
imp=mice(df)
na_index=which(is.na(df$overall_tumor_grade))
complete(imp)$overall_tumor_grade[na_index]  #This can never be 'X'

Thank you for your help and please let me know if you need more information.

New addition

@longrob suggested I temporarily remove the patients with X observations, impute, then add them back in to the full dataset. Would the imputation lose power by removing all of those observations? Along @longrob's suggestion, here is the work-around I have right now using a sample dataframe that is closer to what I'm really working with (several columns with missing values):

df=data.frame(age=rnorm(mean=45,sd=10,25),
overall_tumor_grade=factor(sample(c(1,2,3,'X'),25,replace=TRUE)),
tumor_size=runif(25)*10)
df[df$overall_tumor_grade=='X','tumor_size']=0 #Patients with Grade 'X' have tumorsz=0
df[sample(1:25,3),'age']=NA   ##Setting some observations to NA
df[sample(1:25,5),'overall_tumor_grade']=NA
df[sample(1:25,1),'tumor_size']=NA
################ Imputation
imp=mice(df,meth=c('pmm',"",'pmm'))  #Suppress imputation of overall_tumor_grade
dfimp=complete(imp)
dfimp2=dfimp[dfimp$overall_tumor_grade!='X',]  #I don't want to impute grade 'X' tumors
#so I am trying to remove those observations here and then use droplevels(), but for 
#a reason I can't figure out, that statement is setting all rows to NA where grade='X'

Any suggestions on how I accomplish what @longrob suggested?

Improve this question
$\endgroup$
5
  • $\begingroup$ Can we assume that all instances of overall_tumor_grade == 'X' also have tumor_size == 1? I think the answer might be to construct a data.frame without any rows with 'X', and then do the imputation on that, but that only works if the above assumption is true. $\endgroup$
    – atiretoo
    Commented Jul 12, 2012 at 18:00
  • $\begingroup$ We cannot assume that because by definition grade 'X' means that the tumor size=0 $\endgroup$
    – JJM
    Commented Jul 12, 2012 at 19:25
  • $\begingroup$ @atiretoo I don't see why that assumption is needed ? If there are any observations with tumor_size==0 and overall_tumor_grade==NA just recode those NAs as X. Then delete observations with overall_tumor_grade==NA and run the imputations..... ? $\endgroup$
    – Robert Long
    Commented Jul 12, 2012 at 19:38
  • $\begingroup$ @longrob all of the patients that have tumor sizes of 0 are already coded as grade 'X'. All of the missing overall_tumor_grade observations have tumor sizes >0, and the problem I'm having is that MICE is imputing some of those NAs as grade 'X'. $\endgroup$
    – JJM
    Commented Jul 12, 2012 at 20:08
  • $\begingroup$ Sorry, I just realized there's a typo in my comment, should be tumor size == 0, so your response answers the question, thanks. In your additional code the reason why rows with overall_tumor_grade == NA are set to NA, is that any comparison with NA results in NA, not TRUE or FALSE. You can get around that by picking out the rows with tumor_size == 0, hence my botched question about the data above. $\endgroup$
    – atiretoo
    Commented Jul 12, 2012 at 22:58

2 Answers 2

Reset to default
11
$\begingroup$

The following code defines and calls a dedicated imputation function that separates imputation of cases with tumor_size == 0 from tumor_size > 0. 

## How to impute an ordinal variable with MICE but prevent it from taking one value?

df <- data.frame(age = c(24,37,58,65,70,84),
                 overall_tumor_grade = c(1,1,2,3,'X',NA),
                 tumor_size = c(1.5,2.0,4.2,5.6,0,0.1))

mice.impute.tumor <- function(y, ry, x, ...){
    ymis <- y[!ry]
    tmis <- x$tumor_size[!ry] > 0
        t  <- x$tumor_size > 0
    y[!ry] <- NA
    ymis[!tmis] <- "X"
    ymis[tmis] <- mice.impute.polyreg(y[t, drop = TRUE], ry[t], x[t,], ...)
    ymis
}

ini <- mice(df, maxit = 0)
meth <- ini$meth
meth["overall_tumor_grade"] <- "tumor"
imp <- mice(df, meth = meth, maxit = 1, m = 2)
Improve this answer
$\endgroup$
1
  • $\begingroup$ Versions prior to mice 3.0.0 used the now retired padModel() function to create the design matrix. That function returned a data frame. Since mice 3.0.0 we use base::model.matrix(), which returns a matrix, so the $ operator on x does not work anymore. Patrick Rockenschaub provided an update github.com/amices/mice/issues/224#issuecomment-693935305 $\endgroup$
    – Stef van Buuren
    Commented Sep 17, 2020 at 7:24
7
$\begingroup$

Maybe there is a better way, but the only way I know is to remove the observations with tumor size X from the dataset, then remove the unused level X from the variable definitions (with droplevels() for example) and then run mice()

Update1: OK, there was a typo in my comment last night - I meant to change that line to dfimp2=dfimp[dfimp$overall_tumor_grade!='X' | is.na(dfimp$overall_tumor_grade),] Sorry about that. Anyway, here's my approach to this problem, based on your simulated dataset:

set.seed(100)
df=data.frame(age=rnorm(mean=45,sd=10,25), 
overall_tumor_grade=factor(sample(c(1,2,3,'X'),25,replace=TRUE)), 
tumor_size=runif(25)*10) 
df[df$overall_tumor_grade=='X','tumor_size']=0 #Patients with Grade 'X' have tumorsz=0 
df[sample(1:25,3),'age']=NA   ##Setting some observations to NA 
df[sample(1:25,5),'overall_tumor_grade']=NA 
df[sample(1:25,1),'tumor_size']=NA 

# output the data
df

        age overall_tumor_grade tumor_size
1        NA                   2         NA
2  46.31531                   1  8.5665304
3  44.21083                   1  7.7477889
4  53.86785                   2  8.3402710
5  46.16971                   3  0.9151028
6  48.18630                   2  4.5952549
7  39.18209                   1  5.9939816
8  52.14533                   1  9.1972191
9        NA                   3  9.8282408
10       NA                <NA>  0.3780258
11 45.89886                   2  5.7793740
12 45.96274                   3  7.3331417
13 42.98366                   X  0.0000000
14 52.39840                <NA>  3.0073652
15 46.23380                <NA>  7.3346670
16 44.70683                <NA>  9.0695438
17 41.11146                   2  2.0981677
18 50.10856                   2  3.5813799
19 35.86186                   1  4.4829914
20 68.10297                   3  9.0642643
21 40.61910                   2  3.8943930
22 52.64061                <NA>  5.1745975
23 47.61961                   3  1.2523909
24 52.73405                   X  0.0000000
25 36.85621                   3  7.7180549

# rows 13 and 24 have tumor grade X

# vector for which rows do not have tumor grade X
vx <- df$overall_tumor_grade!='X' | is.na(df$overall_tumor_grade) 
# remove those with tumor grade X
dfimp <- df[vx,]

# do the imputations
dfimp$overall_tumor_grade <- droplevels(dfimp$overall_tumor_grade)
imp=mice(dfimp, printFlag=F) 

# add back the rows we removed
(new_df1 <- rbind(complete(imp,1),df[!vx,]))

1  40.61910                   2  3.8943930
2  46.31531                   1  8.5665304
3  44.21083                   1  7.7477889
4  53.86785                   2  8.3402710
5  46.16971                   3  0.9151028
6  48.18630                   2  4.5952549
7  39.18209                   1  5.9939816
8  52.14533                   1  9.1972191
9  45.89886                   3  9.8282408
10 52.14533                   3  0.3780258
11 45.89886                   2  5.7793740
12 45.96274                   3  7.3331417
14 52.39840                   2  3.0073652
15 46.23380                   3  7.3346670
16 44.70683                   2  9.0695438
17 41.11146                   2  2.0981677
18 50.10856                   2  3.5813799
19 35.86186                   1  4.4829914
20 68.10297                   3  9.0642643
21 40.61910                   2  3.8943930
22 52.64061                   3  5.1745975
23 47.61961                   3  1.2523909
25 36.85621                   3  7.7180549
13 42.98366                   X  0.0000000
24 52.73405                   X  0.0000000
Improve this answer
$\endgroup$
5
  • $\begingroup$ I'm trying to figure out a simple way to do this but I'm hitting a road block because in my data set (i.e. not the sample I posted) there are multiple columns that I am imputing at once. I can suppress imputation of the overall_tumor_grade column while letting the others proceed, but I'm running into trouble trying to then impute overall_tumor_grade while removing the 'X' observations. I'll post some more code in my question to illustrate. $\endgroup$
    – JJM
    Commented Jul 12, 2012 at 20:09
  • 1
    $\begingroup$ @JJM, looking at your additional code, try changing the last line to dfimp2=dfimp[dfimp$overall_tumor_grade!='X' | is.na(df$overall_tumor_grade),] - sorry I don't have time to test this or look in detail, as I have to go to bed now for an early rise (here in UK) but I'll check back on this question tomorrow. $\endgroup$
    – Robert Long
    Commented Jul 12, 2012 at 21:27
  • $\begingroup$ @JJM see my updated answer. Hope it helps ! $\endgroup$
    – Robert Long
    Commented Jul 13, 2012 at 7:22
  • $\begingroup$ Yeah, so that works as long as there are no missing values in other variables in rows with overall_tumor_grade == 'X'. I suppose you could run the imputation a second time after adding the rows back? $\endgroup$
    – atiretoo
    Commented Jul 13, 2012 at 14:10
  • $\begingroup$ @atiretoo , yes, or the other way around (which is what the OP was going to do in the "New addition" section of the question, I think, before running into the problem of selecting on NAs) $\endgroup$
    – Robert Long
    Commented Jul 13, 2012 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.