For example, I trained two models: one with SVM and one with KNN.

Final Prediction = 0.4*KNN + 0.6*SVM

Is this considered blending?

  • $\begingroup$ You can call it ensemble model. $\endgroup$ – Michael M Jul 29 '14 at 13:39
  • $\begingroup$ But there is no official term for this? Is this unheard of? $\endgroup$ – user46925 Jul 29 '14 at 13:48

Yes, you can call that blending. You could also call it a weighted average of two models.

| cite | improve this answer | |
  • $\begingroup$ A side note: Why isn't this technique used widely? Is it because it is hard to find the appropriate weights? $\endgroup$ – user46925 Jul 29 '14 at 14:04
  • $\begingroup$ @user1008537 why do you think it's any good? You lose any theoretical properties of all approaches you're blending and end up with a droopy mess. $\endgroup$ – Marc Claesen Jul 29 '14 at 14:32
  • $\begingroup$ @user1008537 It is widely used. One easy way to find appropriate weights is through cross-validation. You can do a regression on the held-out data to find the weights. This is called a "second-stage model" or "meta model" $\endgroup$ – Zach Jul 29 '14 at 15:22
  • $\begingroup$ @MarcClaesen If your primary modeling goal is predictive accuracy, theoretical properties of individual models is far less important. In this case blending is often an excellent approach: the strengths of one model can make up for the weaknesses of another. $\endgroup$ – Zach Jul 29 '14 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy