1
$\begingroup$

I was wondering if, unlike feed forward networks, a RNN continues to learn when used so the weights are always being optimized even after training has been completed?

$\endgroup$

1 Answer 1

4
$\begingroup$

Actually, this can be done to both FFNN's and RNN's, as long as the correct output is given after your inference. For instance, imagine you have a robot that needs to learn the consequences of its actions on the environment. You could use a FFNN for this task and, after each performed action, the robot will receive the correct output by observing the environment and correct its weights (it is called inverse model learning and you can find a paper which applies RNN's to this kind of task here). Applying this to RNN's on time-series prediction tasks may be more obvious: you predict the next value, then observe the actual value and make corrections. This may or may not be the best approach depending on your problem. For instance, you may have found a good model for your data and don't want to risk messing with it. It is common, for instance, in the videogames industry: if you happen to train (offline) some NN as the controller for an agent, you turn off learning during the game, so it doesn't make something unexpected which breaks the game after release.

UPDATE: testuser suggested citing the term temporal difference learning. It is another great example where you can (in this case you need to) keep learning while acting. It is similar to the robot verifying the consequences of its action, but here it corrects its predictions about the value/quality/reward of the next state.

$\endgroup$
2
  • 1
    $\begingroup$ Good answer, might want to add the term temporal difference learning for those looking for references, though $\endgroup$
    – testuser
    May 27, 2015 at 1:12
  • $\begingroup$ I didn't mention TD specifically in my examples, but it is nice you mentioned it, indeed it is related and useful in the context of the question. $\endgroup$
    – rcpinto
    May 27, 2015 at 1:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.