Are there recurrent neural networks (RNNs) of variable size?
From what I've seen, RNNs are usually built using several nodes (or layers), in a manner similar to unrolled hidden Markov models (HMMs); however, one could construct a Markov model of unspecified size using a single truly recurrent node (or layer), which would have edges back into itself.
One benefit of using this truly recurrent formulation (rather than the unrolled formulation) is that it allows processing of data where the size is not known in advance. This could be used, for example, to feed forward until a node (or layer) variable takes on a certain state, allowing the recurrence to exit. The challenge of the recurrent formulation is that backpropagation may be more complex since we are essentially calculating derivatives within a chain.
Do these truly recurrent NNs exist? I imagine they would be nice for applications in NLP, even if training may be a bit of a pipe dream without resorting to numeric differentiation.