Skip to main content

Suppose you have a Normal Mixture Model with 2 Components - you could write this model as follows:

  • Pi_1 * N(mu_1, sigma_1) + Pi_2 * N(mu_2, sigma_2)

$\pi_1 N(\mu_1, \sigma_1) + \pi_2 N(\mu_2, \sigma_2)$

In the above model, there are 6 unknowns : Pi_1$\pi_1$, mu_1$\mu_1$, sigma_1$\sigma_1$, Pi_2$\pi_2$, mu_2$\mu_2$, sigma_2 $\sigma_2$. In the context of Mixture Models, these 6 unknowns are generally estimated iteratively using the EM (Expectation MaximalizationMaximization) Algorithm.

My Question: What is preventing us from writing the above model in "Likelihood Format" as a function : L (Pi_1, mu_1, sigma_1, Pi_2, mu_2, sigma_2) = .....$L(\pi_1, \mu_1, \sigma_1, \pi_2, \mu_2, \sigma_2) = \dots$ and then jointly estimating these 6 unknowns by either taking the partial derivatives of the likelihood function relative to each of these 6 terms and solving these equations - or using some optimization algorithm like Gradient Descent (if there is no closed form solution)?

Why has the EM algorithm become the standard algorithm for estimating the unknowns of Mixture Models?

Thanks!

Suppose you have a Normal Mixture Model with 2 Components - you could write this model as follows:

  • Pi_1 * N(mu_1, sigma_1) + Pi_2 * N(mu_2, sigma_2)

In the above model, there are 6 unknowns : Pi_1, mu_1, sigma_1, Pi_2, mu_2, sigma_2 . In the context of Mixture Models, these 6 unknowns are generally estimated iteratively using the EM (Expectation Maximalization) Algorithm.

My Question: What is preventing us from writing the above model in "Likelihood Format" as a function : L (Pi_1, mu_1, sigma_1, Pi_2, mu_2, sigma_2) = ..... and then jointly estimating these 6 unknowns by either taking the partial derivatives of the likelihood function relative to each of these 6 terms and solving these equations - or using some optimization algorithm like Gradient Descent (if there is no closed form solution)?

Why has the EM algorithm become the standard algorithm for estimating the unknowns of Mixture Models?

Thanks!

Suppose you have a Normal Mixture Model with 2 Components - you could write this model as follows:

$\pi_1 N(\mu_1, \sigma_1) + \pi_2 N(\mu_2, \sigma_2)$

In the above model, there are 6 unknowns : $\pi_1$, $\mu_1$, $\sigma_1$, $\pi_2$, $\mu_2$, $\sigma_2$. In the context of Mixture Models, these 6 unknowns are generally estimated iteratively using the EM (Expectation Maximization) Algorithm.

My Question: What is preventing us from writing the above model in "Likelihood Format" as a function : $L(\pi_1, \mu_1, \sigma_1, \pi_2, \mu_2, \sigma_2) = \dots$ and then jointly estimating these 6 unknowns by either taking the partial derivatives of the likelihood function relative to each of these 6 terms and solving these equations - or using some optimization algorithm like Gradient Descent (if there is no closed form solution)?

Why has the EM algorithm become the standard algorithm for estimating the unknowns of Mixture Models?

Thanks!

Source Link
stats_noob
  • 1
  • 6
  • 39
  • 124

Estimating Mixture Models with Maximum Likelihood

Suppose you have a Normal Mixture Model with 2 Components - you could write this model as follows:

  • Pi_1 * N(mu_1, sigma_1) + Pi_2 * N(mu_2, sigma_2)

In the above model, there are 6 unknowns : Pi_1, mu_1, sigma_1, Pi_2, mu_2, sigma_2 . In the context of Mixture Models, these 6 unknowns are generally estimated iteratively using the EM (Expectation Maximalization) Algorithm.

My Question: What is preventing us from writing the above model in "Likelihood Format" as a function : L (Pi_1, mu_1, sigma_1, Pi_2, mu_2, sigma_2) = ..... and then jointly estimating these 6 unknowns by either taking the partial derivatives of the likelihood function relative to each of these 6 terms and solving these equations - or using some optimization algorithm like Gradient Descent (if there is no closed form solution)?

Why has the EM algorithm become the standard algorithm for estimating the unknowns of Mixture Models?

Thanks!