0
$\begingroup$

I have a very simple question. My reference textbook is the Murphy, "machine learning, a probabilistic perspective". Let's imagine we are trying to fit a GMM $\gamma$ with MAP.

We know that the label of the point $x_i$ is chosen according to the MAP estimate. $$z_i = argmax_j \: r_{ij} = \log p(x_i|\gamma) + \log (\theta_j) $$ For $n$ points, let $r_j = \sum_i r_{ij}$, where $r_{ij}$ is the responsibility of point $i$ for cluster $j$, and $\alpha_j \in \mathbb{R}$ is just a parameter of the Dirichlet distribution used as prior. $k$ is the number of clusters

Just as an example, the update rules during the fitting part are these:

  • the mixing weights: $$\theta_j = \frac{\sum_{i}r_{ij} +\alpha_j-1}{n + \sum_j \alpha_j - k}$$

  • the centroids: $$ \mu_j^{MAP} = \frac{r_j \mu_j^{ML} + \iota_0 m_0 }{r_j \iota_0}$$ where $m_0$ is the centroid of our prior and $\iota_0$ our confidence

My question is very simple: when i update the model, should i take into account ALL the $n$ points in the dataset when i compute $r_j$ and the parameters or only those points which have been assigned to the cluster $z_i$?

Where actually the initial formula for the labels is used? Only when I need to tell the label of $n$ points the points in my dataset and for the new points?

Thanks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.