I have a very simple question. My reference textbook is the Murphy, "machine learning, a probabilistic perspective". Let's imagine we are trying to fit a GMM $\gamma$ with MAP.

We know that the label of the point $x_i$ is chosen according to the MAP estimate. $$z_i = argmax_j \: r_{ij} = \log p(x_i|\gamma) + \log (\theta_j) $$ For $n$ points, let $r_j = \sum_i r_{ij}$, where $r_{ij}$ is the responsibility of point $i$ for cluster $j$, and $\alpha_j \in \mathbb{R}$ is just a parameter of the Dirichlet distribution used as prior. $k$ is the number of clusters

Just as an example, the update rules during the fitting part are these:

  • the mixing weights: $$\theta_j = \frac{\sum_{i}r_{ij} +\alpha_j-1}{n + \sum_j \alpha_j - k}$$

  • the centroids: $$ \mu_j^{MAP} = \frac{r_j \mu_j^{ML} + \iota_0 m_0 }{r_j \iota_0}$$ where $m_0$ is the centroid of our prior and $\iota_0$ our confidence

My question is very simple: when i update the model, should i take into account ALL the $n$ points in the dataset when i compute $r_j$ and the parameters or only those points which have been assigned to the cluster $z_i$?

Where actually the initial formula for the labels is used? Only when I need to tell the label of $n$ points the points in my dataset and for the new points?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.