# AdaBoost learning rate calculation

I saw the following in a Random Forrest calculation. My understanding of logarithms is not intuitive, I always have to look them up. It was asked:

Calculate this decision tree’s weight in the ensemble
the weight of this tree = learning rate * log( (1 — e) / e)


It was in middle of an Adaboost tutorial. My question is why is this equation with the log chosen? What special properties does it have that makes a good constant for the learning rate?

Basic Ensemble Learning (Random Forest, AdaBoost, Gradient Boosting)- Step by Step Explained Under the section Adaboost.

• Adaboost and random forest are different algorithms, random forest doesn't have learning rate.
– Tim
Jun 5, 2019 at 3:40

Now regarding AdaBoost using the log ratio of the (sum of the) errors in its weight updates: The error term $$e_t$$ is the sum of the weight $$w$$ of the misclassified points. These weights $$w$$ themselves are scaled such that they sum up to $$1$$. i.e. the term $$e$$ will never be larger than $$1$$. Now, when $$e_t$$ is very small, i.e. the tree is a good classifier, AdaBoost wants that tree to be assigned more weight than if it were a bad classifier. For example, assuming we have a strong classifier with $$e_t \leq 0.01$$ the corresponding log ratio term equals $$\log(99)$$ (around $$4.6$$), while if $$e_t \approx 0.5$$ the corresponding ratio is approaches $$\log(1)$$, i.e $$0$$. We need to note here that practically $$e_t$$ is assumed to never get above $$0.5$$; if a base classifier had a $$e_t$$ above $$0.5$$, we would invert the signs of that base classifier and get another with a smaller $$e_t$$. And this brings us to why we picked the term $$\log(\frac{1-e_t}{e_t})$$ as our scaler for the learning rate: it allows us to adapt the learning rate per base classifier. If the base classifier tree is strong (i.e. $$e_t$$ is low) then a large step is taken. If the base classifier tree is bad (i.e. $$e_t$$ approaches $$0.5$$) then a much smaller step is taken. With each step we update our predictions; this means that a "large step" potentially changes our predictions for the better while a "small step" does not significantly deteriorate our predictions.