I'm new to ensemble model. Suppose I've KNN models like this - (in R) library(class)

pred.y1 <- knn(train.x,test.x,k=3)
pred.y2 <- knn(train.x,test.x,k=5)
pred.y3 <- knn(train.x,test.x,k=6)
pred.y4 <- knn(train.x,test.x,k=8)

Can I build an ensemble model out this different results out of these pred.y1, pred.y2, pred.y3, and pred.y4? If so how I can I achieve it?

Related to the same question, for the same dataset, I was building naiveBayes model with e1071 package, and it appears naiveBayes compensates what KNN lacks, and it may be ideal if I combine KNN and naiveBayes results with ensemble model.

Now, suppose, if I've something like this,

model <- naiveBayes(train.x,train.y,laplace = 0.6)
pred.yNB <- predict(model,test.x)

Now, how do I build an ensemble model here with pred.yNB and pred.y1, pred.y2, pred.y3,pred.y4?

Are there any packages in R that can be used for building ensemble models independent of base (prior) modeling technique?

Thank you in advance.


1 Answer 1


Ensemble models in their simplest form are just averaging (regression) or voting (classification) a prediction using multiple models.

So for your example you could average the prediction vector of all 5 models. This is a valid ensemble model.

This paper has an example of using the package ipred with generic models to improve the performance of an ensemble by using Bagging.

You may want to look at Boosting which is the sort of optimization you describe when saying "naiveBayes compensates what KNN lacks".


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.