I'm currently working on implementing the Transformer model for machine translation. I'm taking a look at the data that was used in the actual paper and also used by many other implementations available publicly. The dataset is an English-German dataset (specifically the IWSLT 16 En-De dataset) and it's apparently supposed to be a parallel corpus.
What I don't understand is that doesn't "parallel" in this context mean that for every English sentence there is one German sentence? The dataset contains 233,213 English sentences and 196884 German sentences. Doesn't this mean for some German sentences there will be more than one corresponding English translation?
I'm asking this question because I'm trying to preprocess the data and create batches to feed into a neural network model. The problem is that right now the code I've written simply slices the English dataset up to the length of the German dataset (so that they have the same number of sentences), but my concern is that the data won't actually be parallel.