I'm implementing Q-learning for an RL problem, but most of the rewards are 0 (with the exception of a +1/-1 at the end of the episode). Does Q-learning still make sense in this context?
I'm also curious as to why people use Q-learning in the first place, instead of simply computing the actual discounted reward for each state and targeting the function approximator at that instead. If you're learning happens offline, you have the entire series of state-action-reward triplets, so computing the discounted reward should be possible (albeit slower).