Find variables selected for each subset using caret feature selection

Question

I am doing feature selection using the command 'rfe' in the caret package (http://caret.r-forge.r-project.org/featureselection.html). This command uses a metric to find the optimal amount of variables and which variables that is. However, I would like to also see the other steps in the feature selection than simply the last one. For instance, I would like to know which variables were the optimal ones if I wanted exactly 10 variables.

My code is the following:

ctrl <- rfeControl(functions = rfFuncs,
                   method = "cv",
                   verbose = FALSE)
subsets <- c(5,10,15,20,25)
lmProfile <- rfe(dat2_X, dat2_Y,
                 sizes = subsets,
                 rfeControl = ctrl)

Answer 1

See lmProfile$variables. It has the ranking metrics for each predictor at each iteration. For example, from ?rfe:

data(BloodBrain)

x <- scale(bbbDescr[,-nearZeroVar(bbbDescr)])
x <- x[, -findCorrelation(cor(x), .8)]
x <- as.data.frame(x)

set.seed(1)
lmProfile <- rfe(x, logBBB,
                 sizes = 10:20,
                 rfeControl = rfeControl(functions = lmFuncs, 
                                         number = 15))

head(lmProfile$variables) has:

Overall            var Variables   Resample
4.930084     vsa_other        71 Resample01
4.696723    slogp_vsa5        71 Resample01
3.877510         pnsa1        71 Resample01
3.649555      vsa_base        71 Resample01
3.586327 frac.cation7.        71 Resample01
3.301325        a_base        71 Resample01

For each resample, there are 71 rows here that are the variables selected for a subset size of 71, 20 rows for the ones selected at 20 etc.

Max

Answer 2

Just to complete Max's answer, here is one way to ensure that you select variables of your choice of subset-size from the output given above. Suppose you want a set-size of 10:

set.size = 10 #you want set-size of 10
lm.vars <- lmProfile$variables 

lm.set <- lm.vars[lm.vars$Variables==set.size,  ] # selects variables of set-size (= 10 here)

The issue is that in different folds, different variables may be selected in the set of 10. So we average the importance/ranking score of all variables in set, and choose the highest ranked 10 variables

#use aggregate to calculate mean ranking score (under column "Overall")
lm.set <- aggregate(lm.set[, c("Overall")], list(lm.set$var), mean)

#order from highest to low, and select first 10:
lm.order <- order(lm.set[, c("x")], decreasing = TRUE)[1:set.size]
lm.set[lm.order, ]

This will give the following output:

                Group.1           x
9              hardness 452.3865743
10                 homo 363.2872843
11                 lumo 283.3329715
20                pnsa1   1.0754635
29             smr_vsa0   1.0078744
13 most_positive_charge   0.9588959
6                 fnsa1   0.9310444
14                o_sp2   0.8874429
1                 a_acc   0.8706394
18         peoe_vsa.5.1   0.8072423