Conceptual questions on ensemble learning and Boosting methods in Matlab

Question

The documentation on ensemble methods in Matlab explains different ensemble algorithms for classification and regression tasks. I have normalized the raw feature set and using the normalized data for training and testing.

My data is imbalanced and so I am interested to apply RUSBoost method. It is specifically used for such a scenario. There are some points which are not clear from the documentation and for which I could not find answers in other resources. Few points on ensemble learning cleared from this question on SO

However, I have confusions on applying boosting method with ensemble learning. Here are my questions:

1. Can RUSBoost be applied without using Ensemble classifiers? Are boosting techniques- undersampling and oversampling methods applicable for ensemble classifiers only?
2. Why only decision tree can be used with RUSBoost. I tried using other learners such as knn and svmtemplate but these throw errors.

Please correct me where I have misunderstood.

Answer 1

Regarding question 1:

The RUSBoost implementation is part of the ensemble framework, so it is necessarily tied to ensembles. However, the idea behind RUSBoost can be applied without an ensemble.

Quoting from the description of RUSBoost

RUS stands for Random Under Sampling. The algorithm takes N, the number of members in the class with the fewest members in the training data, as the basic unit for sampling. Classes with more members are under sampled by taking only N observations of every class. In other words, if there are K classes, then, for each weak learner in the ensemble, RUSBoost takes a subset of the data with N observations from each of the K classes.

If we choose to ignore the boosting aspect of the implementation, this is simply applying undersampling to the bigger classes in your data, which is a known concept for tackling class imbalance and is independent of the classifier that you use. Note, however, that this procedure of course leads to less data being used by the classifier, which might impact the predicition quality.
Another approach to undersampling the other classes would be oversampling the minority class(es) , but as always each method has their drawbacks.

Regarding question 2:
From the documentation page about the ensemble learning framework

Except for Subspace, all boosting and bagging algorithms are based on decision tree learners. Subspace can use either discriminant analysis or k-nearest neighbor learners.

This is a restriction of Matlabs implementation, which makes sense because decision trees are very often used in ensemble methods. Some reasons are explained in this answer.