0
$\begingroup$

https://www.cs.mcgill.ca/~dprecup/courses/RL/Lectures/2-bandits-2019.pdf In above pdf document, page 19, they explain by formula: $$\sum _{ b }^{ }{ \frac { \nabla { \Pi }_{ t }(b) }{ \nabla { H }_{ t }(a) } } =\quad 0$$

As the title why it is equal to 0?

$\endgroup$
2
$\begingroup$

It is equal to zero because $\pi$ is a probability distribution over action space. Total probability over action space has to be $1$. If you increase probability for a single action $b$, then probability has to decrease for a set of some other actions, otherwise their total sum would be larger than $1$. The total gradient for all actions has to be zero because if it wasn't zero you would increase/decrease some probabilities and you would not compensate that by decreasing/increasing some other probabilities so their total sum would not be $1$ anymore (it would be larger or lower than $1$ depending if we increased or decreased some probabilities).

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.