I want to evaluate how well a recurrent neural network I've created captures long-range dependencies, and the effects altering the network have on this.
I'm not entirely sure how I would go about doing this except by training it on data where I already have an understanding of the underlying dependencies. As such, I'd like a way to simulate data where I can control the underlying long range dependencies of the data.
To be honest, I'm quite new to deep learning so I'm still getting my head around what I'm actually doing. However, I've read that "long memory" can be defined in terms of autocorrelations; that is, that the autocorrelations decay to zero very slowly. My idea was to generate sequences that have this property, train the network on it and then let the network come up with sequences of it's own, which one would hope would retain this property.
I just don't know how I would go about generating such data!