In the paper introducing DQN "Playing Atari with Deep Reinforcement Learning", it mentioned:
Note that when learning by experience replay, it is necessary to learn off-policy (because our current parameters are different to those used to generate the sample), which motivates the choice of Q-learning.
I didn't quite understand what it means. What if we use SARSA and remember the action
a' for the action we are to take in
s' in our memory, and then sample batches from it and update Q like we did in DQN? And, can actor-critic methods (A3C, for specific) use experience replay? If not, why?