0
$\begingroup$

Suppose I have a task X which is perfectly solvable by a Feed-Forward Neural Network.

When I apply a Recurrent Neural Network to the same problem, may it "loose" it's internal loop-/backward-links (setting them to 0-weight during learning), basically becoming a Feed-Forward Neural Network?


Is a Recurrent Neural Network with very few or very low-weighted backward-links an indicator, that a Feed-Forward Network might solve the problem just as well, with fewer links to compute (no backward-links need to be considered)?

$\endgroup$
  • $\begingroup$ Usually RNN would have much less parameters then dense network, so could you clarify what kind of architecture you have in mind? $\endgroup$ – Tim Jul 22 at 8:58
1
$\begingroup$

When I apply a Recurrent Neural Network to the same problem, may it "loose" it's internal loop-/backward-links (setting them to 0-weight during learning), basically becoming a Feed-Forward Neural Network?

It really depends on what your training algorithm is doing, but generally the answer to your question is yes.

In the absence of more information regarding the topologies your are comparing when referring to recurrent and feed-forward neural networks, a Recurrent Neural Network is a topological superset of a Feed-forward network.

However, in practice, neural networks must be trained. This is effectively a curve-fitting exercise (or an optimisation problem) and is at risk of overfitting. By using a Recurrent network instead of a feed-forward network to solve problems perfectly suited to the former, you are increasing the degrees of freedom in your model and, therefore, the risk of overfitting.

An RNN might therefore be less apt at solving a problem than a feed-forward network (with a simpler topology).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.