a document is translated according to the probability distribution $p(e|f)$ that a string $e$ in the target language (for example, English) is the translation of a string $f$ in the source language (for example, French).

According to the Baye's Theorem based modeling of the the probability distribution $p(e|f)$, $$ p(e|f) \propto p(f|e) p(e) $$

where the translation model $p(f|e)$ is the probability that the source string is the translation of the target string, and the language model $p(e)$ is the probability of seeing that target language string.

$$ $$

Finding the best translation $\tilde {e}$ is done by picking up the one that gives the highest probability: $\tilde{e} = arg \max_{e \in e^*} p(e|f) = arg \max_{e\in e^*}p(f|e) p(e)$.

($e^*$ denotes all the strings in the target language)

Now, I have two questions.

  1. How is $p(f|e)$ calculated?
  2. If it is calculated from the parallel corpus, why can't we also calculate $p(e|f)$ directly from the corpus?
  • $\begingroup$ Here's a response relevant to the subject title of your question. Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (arxiv.org/abs/1907.05019) provides one explanation. $\endgroup$
    – user234562
    Nov 21 '20 at 17:04
  • 1
    $\begingroup$ @user332577 I skimmed through the paper, but I didn't find any explanation for my doubts. $\endgroup$ Nov 21 '20 at 19:38
  • $\begingroup$ As noted in my comment, I'm not responding to your specific doubts but to the more general subject in your title. $\endgroup$
    – user234562
    Nov 22 '20 at 17:22

The reason is not that computing $p(e|f)$ would not be possible. The reason for the factorization is that you want to bring in the language mode $p(e)$ that can be used in decoding. At the time, SMT was invented, $n$-gram language models were quite good and it would be a pity not to use them.

The probability $p(f|e)$ is factorized over phrases (in fact frequent n-grams). The phrase pairs and their probabilities are stored in a so-called phrase table. The phrase table is obtained by computing word alignment on the parallel corpus (which gives you probability scores of words being aligned) and joining the words into more common n-grams. So, at this stage, the difference between $p(e|f)$ and $p(f|e)$ is how you normalize the scores.

Computing the conditional probability in this way tells you how to score a sentence pair (look-up phrases in the tables and multiply their scores), but it does not really tell you how to decode a sentence. This is why the language model comes into play. For a source sentence, you pick up some candidate phrases (scored with the reversed probability direction) and using the language model scores, you try to build a sentence as fluent as possible.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.