Skip to main content
added 41 characters in body
Source Link
Xi'an
  • 107.7k
  • 13
  • 190
  • 676

From the book "Machine Learning a probabilistic Perspective""Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated:

The usual representation (of a finite mixture model) is as follows:

$p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\theta_k)$

$p(z_i = k| \boldsymbol\pi = \pi_k) = \pi_k$

$p(\boldsymbol\pi|\alpha) =\text{Dir}(\boldsymbol\pi|(\alpha/K)\boldsymbol1_K)$

The form of $p(\boldsymbol\theta_k|\lambda)$ is chosen to be be conjugate to $p(x_i|\boldsymbol\theta_k)$. We can write $p(x_i|\boldsymbol\theta_k)$ as $\boldsymbol{x}_i \sim F(\boldsymbol\theta_{z_i})$ where F is the observation distribution. Similarly, we can write $\boldsymbol\theta_k \sim H(\lambda)$, where H is the prior.

Now this modelling is quite confusing to me. What is the difference between $\boldsymbol\theta_k$ and $\boldsymbol\theta_{z_i}$? What is meant by "Observation distribution"? Can we apply EM algorithm to this model, how?

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated:

The usual representation (of a finite mixture model) is as follows:

$p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\theta_k)$

$p(z_i = k| \boldsymbol\pi = \pi_k) = \pi_k$

$p(\boldsymbol\pi|\alpha) =\text{Dir}(\boldsymbol\pi|(\alpha/K)\boldsymbol1_K)$

The form of $p(\boldsymbol\theta_k|\lambda)$ is chosen to be be conjugate to $p(x_i|\boldsymbol\theta_k)$. We can write $p(x_i|\boldsymbol\theta_k)$ as $\boldsymbol{x}_i \sim F(\boldsymbol\theta_{z_i})$ where F is the observation distribution. Similarly, we can write $\boldsymbol\theta_k \sim H(\lambda)$, where H is the prior.

Now this modelling is quite confusing to me. What is the difference between $\boldsymbol\theta_k$ and $\boldsymbol\theta_{z_i}$? What is meant by "Observation distribution"? Can we apply EM algorithm to this model, how?

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated:

The usual representation (of a finite mixture model) is as follows:

$p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\theta_k)$

$p(z_i = k| \boldsymbol\pi = \pi_k) = \pi_k$

$p(\boldsymbol\pi|\alpha) =\text{Dir}(\boldsymbol\pi|(\alpha/K)\boldsymbol1_K)$

The form of $p(\boldsymbol\theta_k|\lambda)$ is chosen to be be conjugate to $p(x_i|\boldsymbol\theta_k)$. We can write $p(x_i|\boldsymbol\theta_k)$ as $\boldsymbol{x}_i \sim F(\boldsymbol\theta_{z_i})$ where F is the observation distribution. Similarly, we can write $\boldsymbol\theta_k \sim H(\lambda)$, where H is the prior.

Now this modelling is quite confusing to me. What is the difference between $\boldsymbol\theta_k$ and $\boldsymbol\theta_{z_i}$? What is meant by "Observation distribution"? Can we apply EM algorithm to this model, how?

Source Link

Confusion in modelling finite mixture model

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated:

The usual representation (of a finite mixture model) is as follows:

$p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\theta_k)$

$p(z_i = k| \boldsymbol\pi = \pi_k) = \pi_k$

$p(\boldsymbol\pi|\alpha) =\text{Dir}(\boldsymbol\pi|(\alpha/K)\boldsymbol1_K)$

The form of $p(\boldsymbol\theta_k|\lambda)$ is chosen to be be conjugate to $p(x_i|\boldsymbol\theta_k)$. We can write $p(x_i|\boldsymbol\theta_k)$ as $\boldsymbol{x}_i \sim F(\boldsymbol\theta_{z_i})$ where F is the observation distribution. Similarly, we can write $\boldsymbol\theta_k \sim H(\lambda)$, where H is the prior.

Now this modelling is quite confusing to me. What is the difference between $\boldsymbol\theta_k$ and $\boldsymbol\theta_{z_i}$? What is meant by "Observation distribution"? Can we apply EM algorithm to this model, how?