I am quite new to audio signal processing, more specifically speaking speaker verification. I have trained a CNN-based Siamese network to do speaker verification. The whole thing is trained with one dataset, in which 720 person's voice clip pairs after VAD and framing are selected for training. By the end of training, the model achieved the accuracy of 85% and 83% on the training and validataion data sets, respectively. with 1000 voice clip pairs of 15 strangers in the same dataset, the model still achieved the accuracy of 82%. However, when 10000 voice clip pairs of 100 strangers in another dataset is employed for test, only 68% accuracy is delivered by the model.
I'd like to point out that:
1.for both training and test phases, voice clips from both datasets are clean and without any noise.
2.by visually judging the difference of voice clips from different datasets, the voice clips from the dataset for training normally have more intensive amplitude in time domain, while those form the dataset for test got weaker amplitudes(see the figure below as a clue). maybe that makes samples for training and test not complying with the condition of i.i.d(independent and identically distributed)?
So, based on the problem described above, are there any guru to help me point out where the problem is? and any strategy to eliminate it?
If such a problem could be fix with a free and open sourced speaker dataset on internet, I would buy it. Otherwise, I would lean to solutions with limited dataset.
Thanks in advance, I really appreciate it!