# Why can't we calculate p(e|f) directly from corpus?

In SMT,

a document is translated according to the probability distribution $$p(e|f)$$ that a string $$e$$ in the target language (for example, English) is the translation of a string $$f$$ in the source language (for example, French).

According to the Baye's Theorem based modeling of the the probability distribution $$p(e|f)$$, $$p(e|f) \propto p(f|e) p(e)$$

where the translation model $$p(f|e)$$ is the probability that the source string is the translation of the target string, and the language model $$p(e)$$ is the probability of seeing that target language string.



Finding the best translation $$\tilde {e}$$ is done by picking up the one that gives the highest probability: $$\tilde{e} = arg \max_{e \in e^*} p(e|f) = arg \max_{e\in e^*}p(f|e) p(e)$$.

($$e^*$$ denotes all the strings in the target language)

Now, I have two questions.

1. How is $$p(f|e)$$ calculated?
2. If it is calculated from the parallel corpus, why can't we also calculate $$p(e|f)$$ directly from the corpus?
• Here's a response relevant to the subject title of your question. Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (arxiv.org/abs/1907.05019) provides one explanation.
– user234562
Nov 21 '20 at 17:04
• @user332577 I skimmed through the paper, but I didn't find any explanation for my doubts. Nov 21 '20 at 19:38
• As noted in my comment, I'm not responding to your specific doubts but to the more general subject in your title.
– user234562
Nov 22 '20 at 17:22

The reason is not that computing $$p(e|f)$$ would not be possible. The reason for the factorization is that you want to bring in the language mode $$p(e)$$ that can be used in decoding. At the time, SMT was invented, $$n$$-gram language models were quite good and it would be a pity not to use them.
The probability $$p(f|e)$$ is factorized over phrases (in fact frequent n-grams). The phrase pairs and their probabilities are stored in a so-called phrase table. The phrase table is obtained by computing word alignment on the parallel corpus (which gives you probability scores of words being aligned) and joining the words into more common n-grams. So, at this stage, the difference between $$p(e|f)$$ and $$p(f|e)$$ is how you normalize the scores.