Let us have a set of languages $L = \{l_{i=1}, \cdots, l_N\}$. Let us have a set of words $W = \{w_{j=1}, \cdots, w_M\}$ such that each word $w_j$ belongs to a language $l_i$. Let $n_i$ denote how many words each language $l_i$ has.

Let $W$ and $L$ be Discrete Random Variables so we can calculate the Posterior Probability $P(L|W)$ by using Bayes`s Theorem:

$P(L|W) = \frac{P(W|L)P(L)}{P(W)}$,

where P(W|L) is the Likelihood of $W$ given $L$, $P(L)$ is the Prior Probability of $L$, and $P(W)$ is the Marginal Probability of $W$.

How do I calculate $P(W|L)$ and $P(W)$?


The likelihood $P(W|L)$ typically comes from what sort of model/relationship you are assuming between the variables in your data. For example in simple linear regression we have $$ p(\beta | \mathbf{y}) \propto p(\mathbf{y} | \beta)p(\beta) $$ where $p(\mathbf{y}|\beta)$ is assumed to be be $N(\mathbf{X}\beta, \sigma^2)$.

The marginal likelihood $P(W)$ is often difficult or even intractable to calculate. In some simple models it can be calculated. Many inferential procedures however do not require you to be able to calculate this value.

  • $\begingroup$ ok, let's ignore P(W)... I still don't understand P(W|L).. The model/relationship I assumed was that each word belongs to a language, but I still don't get it. $\endgroup$ May 23 '17 at 16:21
  • $\begingroup$ I don't know anything about your problem, nor have you given me enough information to provide you with any reasonable information about the likelihood. $\endgroup$
    – bdeonovic
    May 24 '17 at 1:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.