Are there any reasons why one wouldn't want to use certain algorithms together in stacking?

For example, if I decided to use an SVM and a LogisticRegression classifier and were considering adding a third one to my lower layer, would I have any reason to avoid certain algorithms from a purely mathematical/theoretical perspective?

Clearly, I want to choose an algorithm that performs well on the instances that were incorrectly labeled by the other two algorithms, but is that the only driver for considering different hypothesis classes?

  • $\begingroup$ Hard to answer as you generally do not know such things in advance, the only thing that could help you is whether you know what the relationship between the variables is like, so for ex. if you have clusters you would not use regression. $\endgroup$ – user2974951 Jan 10 at 7:53

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.