0
$\begingroup$

While the term 'appropriate' is usually subjective and largely dependent upon the domain that one is talking about, I am here asking whether the exclusion of outliers goes against the principle of ensemble learning.

For instance, in an information retireval system that combines multiple weak learners in the hope of producing an overall good result, would it be considered inappropriate in an academic setting for outliers, that do not have statistical significance, to be excluded from each of the learners prior to combination? Or, to put it another way, would the exclusion of outliers at each step of an ensemble process defeat some of the purpose of an ensemble method?

The importance of this decision lies in the weighting that would be applied to the different learners. A learner with a high occurrence of outliers will be negatively effected in terms of its weighting.

$\endgroup$
1
$\begingroup$

would the exclusion of outliers at each step of an ensemble process defeat some of the purpose of an ensemble method?

No, it would not. The idea of an ensemble method is just to pool the strengths of multiple models (possibly multiple iterations of pretty much the same model, as in a random forest) by combining the models. If none of the models can handle outliers itself, it may still be useful to have an outlier-removal step. It all depends on what you're trying to do with the ensemble and how you think the outliers came about.

outliers, that do not have statistical significance

To be clear, statistical significance is a judgment you make about an effect on the basis of a significance test. It doesn't make sense to speak of a data point being significant or not.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.