What is momentum-like training aid technique for reinforcement learning (Q learning)? Is there any method that could help a reinforcement learning (specifically Q learning) model converge?
Can optimization strategy like momentum/Adam/RMSProp applied to Q learning to update Q-table value?
$Q(S, A) ← (1-α)*Q(S,A) + α * [reward + γ * maxQ(S',a)]$
 A: 
Can optimization strategy like momentum/Adam/RMSProp applied to Q learning to update Q-table value?

These optimisers are all based around gradient estimates for gradient descent/ascent, working with gradients across vectors of parameters. The updates to a Q table cannot benefit from this, they are not based around gradient estimates, and are updates to isolated scalar values.
There is no direct equivalent to any of these techniques that would take the individual return estimates (the TD target) from Q-learning and optimise towards faster convergence in a general sense.
If you stay with a tabular approach, there are some things that might help:


*

*Increase the learning rate. In a deterministic, or close-to-deterministic environment, and only for tabular learning, you can set $\alpha$ high, even to $1$ , and the algorithm will still work fine. 

*Initialise Q using a heuristic. If it is possible to calculate and scale a rough heuristic quickly and insert into the Q table, this prior value should guide initial policy and speed up learning.

*Use experience replay. You can store $(S,A,R,S')$ and re-use them in batches on every step. You must re-calculate the TD target each time you re-use the sampled reward and transition data (i.e. don't store TD target, refer the Q table just in time when you need to find $maxQ$). This could apply in your case as you seem to want to get the best convergence over the least time steps, and don't mind extra computation for each time step.

*Use multi-step returns, with eligibility traces. These are effective when learning from sparse rewards, as on each update multiple previously-seen state/action pairs are updated based on latest samples. The Q-learning variant for this is called Q($\lambda$).
You can use these ideas in combination, if all apply (e.g. if you have a deterministic environment with sparse rewards, and there is a simple heuristic available).
Finally, although you said this was for a tabular solver, you can also consider:


*

*Use a function approximator. If you reduce the state representation down to a descriptive vector, you can benefit from generalisation effects of learning using e.g. a neural network. Unlike a tabular estimate, if a neural network estimates the return for a Q value in a state, action it has not seen before, it may be able to generalise to a reasonable first estimate. When you use a neural network, you will also be able to use momentum/Adam/RMSProp etc for the updates to the network (although this won't necessarily make the whole thing faster). You also must use experience replay if you go this route, otherwise Q-learning can be unstable.


There's quite a lot to get to grips with expanding from tabular to neural-network-based approaches, but it is still a possible answer to improved convergence, especially if your Q table is very large (tens of millions of entries or larger).
In theory you could train a neural network purely as a means to populate unseen $(s,a)$ pairs for a first estimate when they are encountered. I would expect that could help with convergence in a similar manner to adding an initial heuristic, although I have not explored this option myself.
