For machine translation purposes I use bleu score, which seems to be the validation mechanism of choice (used in the sutskever 2014 sequence-to-sequence).
The purpose is to get as high bleu as possible (between 0 to 1).
The following mumble gives an extraordinary high bleu score (0.77):
from nltk import bleu_score reference = ['The moon is very bright'] hypothesis = ['Dee dd ss eee'] reference = [[r.split()] for r in reference] hypothesis = [[h.split()] for h in hypothesis] bleu_score.corpus_bleu(reference, hypothesis)
Why does the bleu score give such high accuracy for mumbling? Which other tools for validation could I use for machine translation?