0
$\begingroup$

I watched a lot of video lectures on reinforcement learning with neural networks and have been making some notes from them.

One of the lectures briefly discussed something called parameter or network switching.

From memory, it involved training two neural networks in parallel, with different sets of parameters, with the purpose being that the resulting networks would provide more stable results.

Unfortunately, I can't find anything else on the topic in Google, and can't find which video lecture mentioned this. I remember that it was mentioned in the same lecture as Experience Replay.

Where can I find more information on parameter/network switching? I think I might have written down the wrong words as typing those words into Google don't give me relevant results.

$\endgroup$
0
1
+100
$\begingroup$

Regular DQN always has two networks since that's the way Q learning works and this provides more stable results than a Bellman Residual Minimization based approach (i.e. simply taking the gradient of the temporal difference error).

Double Q learning is a variation of Q learning that keeps two estimates of Q and switches between them. But when used in the context of deep RL, it usually works a little bit different. The essential insight is that, when computing the target Q values, it can be beneficial to use different estimates of Q to select the next action and to determine the value of that action. When used with DQN, those two estimates may just be the online network and the target network but perhaps this is what you mean.

van Hasselt, Hado, Guez, Arthur, and Silver, David. Deep Reinforcement Learning with Double Qlearning. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 2016. http://arxiv.org/abs/1509.06461.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.