I am working on a specific implementation of bootstrap aggregating (bagging). I want to see how well this bagging works for different base classifiers. But so far, the decision tree seems to be the only option that I'm running into. I tried Naive Bayes but bagging seems to make things worse for some data sets. Can you suggest some more algorithms that could be used as base learner for bagging?
Basically you can bag any base learner. Following diversification idea, they should tend to have low bias (= overfit!) and, consequently, high variance.
Examples would include:
- deep decision trees
- $k$-nearest neighbour with low $k$
- richly parametrized linear models (e.g. with splines, pairwise interactions etc.)
- neural nets with enough complexity
Not very suitable from this perspective are
- tree stumps
- too simple linear models
Bagging build new models using the same classifier on variants of the data set. If the classifier is very stable, the models will have a lot of agreement and you won't gain too much from the bagging. The less stable the classifier, the more likely you will gain.
In the original paper about bagging, Breiman refers to this point.
Unstability was studied in Breiman  where it was pointed out that neural nets, classication and regression trees, and subset selection in linear regression were unstable,while k-nearest neighbor methods were stable.
Breiman  is "Breiman,L.(1994)Heuristics of instability in model selection,Technical Report, Statistics Department, University of California at Berkeley."
So decision trees are indeed good candidate classifier for bagging but so are the classifiers listed above and other unstable classifiers. Not that stability is also a function of the data itself.
For more information see here.