I have generated an adaboost classifier in Weka on a dataset where each instance falls into one of two classes. The result was a number of decision trees, each assigned a weight.

What is the proper method for implementing the classifier generated by adaboost? I assumed the answer was

(Weight of Tree 1 * decision of tree1) + (weight of tree 2 * decision of tree 2) ... + (weight of tree n * decision of tree n)

Where each tree will decide if the instance falls into class A (returning +1) or class B (returning -1)

If the final result sum of weight*result is positive the instance is class A, if negative, class B.

The problem is when I implemented exactly this the results are nowhere near what Weka produced, so I assume that this was not the correct way to implement the classifier.

What should I have done instead?

• I don't know what Weka does, but usually Adaboost builds a tree, reweighs the data based on the outcomes and uses the reweighed data to build the next tree. This process is repeated. In addition, at every state only a (typically small) multiple of the new tree is added to the answer. Such a process would not produce the formula you describe. – meh Apr 15 '16 at 21:54
• It looks like you have accidentally created a second account and that is stopping you from being able to immediately edit your own question. Have a look at our help centre to see how to merge them. – Silverfish Apr 15 '16 at 22:01
• -aginensky: That's exactly what happened, but I'm not asking a question about how to implement adaboost, but rather how to implement the classifier that adaboost produced. The classifier is presented as a series of trees, each with an assigned weight. Here is an example: youtu.be/ix6IvwbVpw0?t=343 Here we see a number of weighted classifiers. Combine the weights * classifier and check the sum. – djc6535 Apr 15 '16 at 22:17

Let $G_m(x) \ m = 1,2,...,M$ be the sequence of weak classifiers, the objective is to build the following:
$$G(x) = \text{sign} \left( \alpha_1 G_1(x) + \alpha_2 G_2(x) + ... \alpha_M G_M(x)\right) = \text{sign} \left( \sum_{m = 1}^M \alpha_m G_m(x)\right)$$
Here the coefficients $\alpha_m$ are computed by the boosting algorithm, and weight the contribution of each respective $G_m(x)$.