In Richard S. Sutton and Andrew G. Barto's book on reinforcement learning on page 156 it says:
Maximization bias occurs when estimate the value function while taking max on it (that is what Q learning do), and maximization may not take on the true value which may introduce bias.
- Why can double Q learning solve this problem, and what is the proof for that?
- Does maximization bias always underestimate or always overestimate the true value? Why?