I would like to create an ensemble classifier for a dataset and use different classification models for different subsets of features (these feature subsets are predefined as the data set I am working on has different modalities that require different algorithms/learners).

In mlr (R) I am aware of bagging (but this does not allow me to specify the feature subsets) and stacking (but this is not allowing me to specify feature subsets). Is there a way to generate such an ensemble in mlr (R)?

df <- iris %>% mutate(Species=as.character(Species)) %>% filter(Species %in% c('setosa', 'versicolor'))
tsk1 <- makeClassifTask(data = df[,c('Sepal.Length', 'Sepal.Width', 'Species')], target = "Species")
tsk2 <- makeClassifTask(data = df[,c('Petal.Length', 'Petal.Width', 'Species')], target = "Species")
lrn1 <- makeLearner('classif.rpart',predict.type = "prob")
lrn2 <- makeLearner('classif.svm',predict.type = "prob")
m1 <- train(lrn1, tsk1)
m2 <- train(lrn2, tsk2)
res1 = predict(m1, tsk1)
res2 = predict(m2, tsk2)
rowMeans(cbind(res1$data$prob.setosa, res2$data$prob.setosa))

I had the same problem. A workaround is to wrap each learner with a filter wrapper that filters a fixed set of predictors using the argument fw.mandatory.feat. See example below using the pid.task that is already integrated in mlr.


#Make a base learner
lrn <- makeLearner('classif.logreg', predict.type = 'prob')

#Make feature sets
blood <- c('glucose', 'insulin')
other <- setdiff(getTaskFeatureNames(pid.task), blood)

#make a learners that only predicts from blood parameters.
#The fw.method does not matter, but I use "variance" here because it is the fastest:
lrn.blood <- makeFilterWrapper(lrn, fw.method = 'variance', fw.abs = length(blood),
                               fw.mandatory.feat = blood) %>%

lrn.other <- makeFilterWrapper(lrn, fw.method = 'variance', fw.abs = length(other),
                               fw.mandatory.feat = other) %>%

lrns <- list(lrn.blood, lrn.other)

#Create a super learner
sl <- makeLearner('classif.plsdaCaret', ncomp = 1)

#make stacked learner
stacked <- makeStackedLearner(lrns, super.learner = sl,
                              method = 'stack.cv', predict.type = 'prob') 

#Perform cross validation
benchmark(c(lrns, list(stacked)), pid.task, cv10, auc, models = F)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.