###LDA does not have a distance metric

The intuition behind the LDA topic model is that words belonging to a topic appear together in documents. Unlike typical clustering algorithms like K-Means, it does not assume any distance measure between topics. Instead it infers topics purely based on word counts.

This can be appreciated from the Gibbs sampler described in paper by Griffiths et al.:

$$ P(z_i=j \mid \textbf{z}_{-i} , \textbf{w} ) \propto \frac{n^{(w_i)}_{-i,j}+\beta}{n^{(.)}_{-i,j}+W\beta} \times \frac{n^{(d_i)}_{-i,j}+\alpha}{n^{(d_i)}_{-i,.}+T\alpha} $$

$P(z_i=j \mid \textbf{z}_{-i} , \textbf{w} )$ refers to the probability of assigning topic $j$ to $i^{th}$ word, given all other assignments. This depends on two probabilities:

1. Probability of word $w_i$ in topic $j$
2. Probability of topic $j$ in document $d_i$

These probabilities can be easily computed using the following counts:

• $n^{(w_i)}_{-i,j}:$ number of times word $w_i$ was assigned to topic $j$
• $n^{(.)}_{-i,j}:$ total number of words assigned to topic $j$
• $n^{(d_i)}_{-i,j}:$ number of times topic $j$ was assigned in document $d_i$
• $n^{(d_i)}_{-i,.}:$ total number of topics assigned in document $d_i$
• $T:$ number of topics
• $W:$ number of words in vocabulary
• $\alpha, \beta:$ Dirichlet hyperparameters

Note that all counts are excluding the current assignment, denoted by the $-i$ subscript.

###Why does LDA work?

Referring to these Video Lectures, David Blei attributes it to the following:

###LDA does not have a distance metric

The intuition behind the LDA topic model is that words belonging to a topic appear together in documents. Unlike typical clustering algorithms like K-Means, it does not assume any distance measure between topics. Instead it infers topics purely based on word counts, based on the bag-of-words representation of documents.

This can be appreciated from the Gibbs sampler described in paper by Griffiths et al.:

$$ P(z_i=j \mid \textbf{z}_{-i} , \textbf{w} ) \propto \frac{n^{(w_i)}_{-i,j}+\beta}{n^{(.)}_{-i,j}+W\beta} \times \frac{n^{(d_i)}_{-i,j}+\alpha}{n^{(d_i)}_{-i,.}+T\alpha} $$

$P(z_i=j \mid \textbf{z}_{-i} , \textbf{w} )$ refers to the probability of assigning topic $j$ to $i^{th}$ word, given all other assignments. This depends on two probabilities:

1. Probability of word $w_i$ in topic $j$
2. Probability of topic $j$ in document $d_i$

These probabilities can be easily computed using the following counts:

• $n^{(w_i)}_{-i,j}:$ number of times word $w_i$ was assigned to topic $j$
• $n^{(.)}_{-i,j}:$ total number of words assigned to topic $j$
• $n^{(d_i)}_{-i,j}:$ number of times topic $j$ was assigned in document $d_i$
• $n^{(d_i)}_{-i,.}:$ total number of topics assigned in document $d_i$
• $T:$ number of topics
• $W:$ number of words in vocabulary
• $\alpha, \beta:$ Dirichlet hyperparameters

Note that all counts are excluding the current assignment, denoted by the $-i$ subscript.

###Why does LDA work?

Referring to these Video Lectures, David Blei attributes it to the following: