# Three doubts about off-policy n-step sarsa algorithm

I'm reading the public draft (pdf) of the 2nd edition of Sutton&Barto's RL Book.

There are a few things I don't understand about the Off-policy n-step Sarsa method described in the book. You can find the pseudo-code at page 159 (177) of the pdf I linked above.

Here's a picture of the pseudo-code for your convenience:

Here are the 3 things I don't understand:

1. $\rho$ considers the actions $A_{\tau+1},\ldots,A_{\tau+n-1}$. Shouldn't it also consider $A_{\tau+n}$?
2. In the third to last line, $\rho$ multiplies both $G$ and $Q(S_\tau,A_\tau)$. Shouldn't it multiply just $G$?
3. The second to last line says that $\pi$, if being learned, must be $\epsilon$-greedy wrt $Q$ (with $\epsilon>0$). I don't understand the reason of this restriction. Can't $\pi$ be greedy just like in Q Learning? The exploration is already taken care of by $\mu$.
• Regarding point 2, I am having the same doubt, I actually asked the question before I found yours: ai.stackexchange.com/questions/11676/… – Antoine Savine Apr 5 '19 at 14:31
• I even messaged Rich Sutton, hopefully we get an answer. – Antoine Savine Apr 5 '19 at 14:31
• Were you able to clarify the question (since you asked yourself the same question two years ago :)? – Antoine Savine Apr 5 '19 at 14:32