# Derivation of gradient-bandit algorithm, Why is the sum of the derivatives is zero?

https://www.cs.mcgill.ca/~dprecup/courses/RL/Lectures/2-bandits-2019.pdf In above pdf document, page 19, they explain by formula: $$\sum _{ b }^{ }{ \frac { \nabla { \Pi }_{ t }(b) }{ \nabla { H }_{ t }(a) } } =\quad 0$$

As the title why it is equal to 0?

## 1 Answer

It is equal to zero because $$\pi$$ is a probability distribution over action space. Total probability over action space has to be $$1$$. If you increase probability for a single action $$b$$, then probability has to decrease for a set of some other actions, otherwise their total sum would be larger than $$1$$. The total gradient for all actions has to be zero because if it wasn't zero you would increase/decrease some probabilities and you would not compensate that by decreasing/increasing some other probabilities so their total sum would not be $$1$$ anymore (it would be larger or lower than $$1$$ depending if we increased or decreased some probabilities).