Can a Recurrent Neural Network degenerate to a Feed-Forward Neural network?

Question

Suppose I have a task X which is perfectly solvable by a Feed-Forward Neural Network.

When I apply a Recurrent Neural Network to the same problem, may it "loose" it's internal loop-/backward-links (setting them to 0-weight during learning), basically becoming a Feed-Forward Neural Network?

---

Is a Recurrent Neural Network with very few or very low-weighted backward-links an indicator, that a Feed-Forward Network might solve the problem just as well, with fewer links to compute (no backward-links need to be considered)?

Answer 1

When I apply a Recurrent Neural Network to the same problem, may it "loose" it's internal loop-/backward-links (setting them to 0-weight during learning), basically becoming a Feed-Forward Neural Network?

It really depends on what your training algorithm is doing, but generally the answer to your question is yes.

In the absence of more information regarding the topologies your are comparing when referring to recurrent and feed-forward neural networks, a Recurrent Neural Network is a topological superset of a Feed-forward network.

However, in practice, neural networks must be trained. This is effectively a curve-fitting exercise (or an optimisation problem) and is at risk of overfitting. By using a Recurrent network instead of a feed-forward network to solve problems perfectly suited to the former, you are increasing the degrees of freedom in your model and, therefore, the risk of overfitting.

An RNN might therefore be less apt at solving a problem than a feed-forward network (with a simpler topology).