I'm new Reinforcement learning and currently reading Sutton & Barto's book "Reinforcement Learning: An Introduction". In Chapter 2, they compare greedy and non-greedy methods on 10-armed bandit problem. On page 29, the following figure is shown
I understand what the upper plot is conveying but I can't wrap my head around on the lower graph. In the text, this is how it is described:
The lower graph shows that the greedy method found the optimal action in only approximately one-third of the tasks. In the other two-thirds, its initial samples of the optimal action were disappointing, and it never returned to it. The $\epsilon$-greedy methods eventually performed better because they continued to explore and to improve their chances of recognizing the optimal action. The $\epsilon$ = 0.1 method explored more, and usually found the optimal action earlier, but it never selected that action more than 91% of the time. The $\epsilon$ = 0.01 method improved more slowly, but eventually would perform better than the $\epsilon$ = 0.1 method on both performance measures shown in the figure.
The questions I have are:
For the greedy case, it says "In the other two-thirds, its initial samples of the optimal action were disappointing, and it never returned to it". What do it the text mean by "it never returned to it" and how is that obvious from the plot?
How is the case where $\epsilon = 0.01$ outperforming $\epsilon = 0.1$?
In general, I'm confused about what's being measured and the explanation of the results.