1
$\begingroup$

As the title implies... For instance, for Machine Translation, we have BLEU. For categorization, we have categorical crossentropy, for binary categorization, we have binary crossentropy, etc. etc.

For machine learning methods, I'm not sure what we can use to measure accuracy of TTS models.

$\endgroup$
  • $\begingroup$ Is it feasible to 'translate' speech' output back to text (using a method of known accuracy) and then do matching of back-translated text with original text? $\endgroup$ – BruceET Jul 30 at 19:22
  • $\begingroup$ Hmph that's an interesting point... but still cascading errors can be unpredictable so I was looking for something more complete. $\endgroup$ – Gust Jul 31 at 8:18
0
$\begingroup$

It is better to start exploring such a complex topic like TTS with a textbook. The book by Paul Taylor is good, it covers speech evaluation too

There are basically two approaches - subjective evaluation and objective evaluation. For subjective evaluation the most popular evaluation metric is MOS (mean opinion score test), but there are other more complicated tests like MUSHRA

For objective evaluations the most popular test is simple MCD test (mel cepstral distortion), but there are more advanced ones. For more details see

Speech synthesis evaluation Sébastien Le Maguer

There are also more advanced machine learning methods for synthesis quality, see this paper:

Towards Signal-Based Instrumental Quality Diagnosis for Text-to-Speech Systems by Tiago H. Falk, Student Member, IEEE, and Sebastian Möller

also

Non-intrusive Quality Assessment of Synthesized Speech using Spectral Features and Support Vector Regression by Meet H. Soni, Hemant A. Patil

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.