I'm looking for something like the scikit-learn sample generators for generating small machine translation datasets that are quick to learn. My goal is to debug a Transformer model and have a quick feedback loop. Ie. I want to make a change, train for thirty seconds, infer for two seconds, look at the result and see what I can do next. With ‘debug’ I mean find mistakes in the code I've written, not determining why neurons are dying.
My current idea is to come up with two simple grammars that have a one-on-one mapping between them, generate examples from them and train the Transformer on these examples. But there should be people who have done this before and know what kinds of grammars, alphabets etc. are good.