I have $N$ observations ($x_1, x_2,.. ,x_N$) from a HMM with $K$ latent states. The M step for computing the observation model $\mu_k$ involves maximizing the expression:

$$ L = \sum_{n=1}^{N}{ln \sum_{k=1}^K {\pi_k \mathcal{N}(x_n|\mu_k,\Sigma_k) }} $$

Setting $\frac{\partial L}{\partial \mu_k}$ to 0, we get

$$ \mu_k = \frac{\sum_{n=1}^N \gamma(z_{nk}) x_n}{\sum_{n=1}^N \gamma(z_{nk})} \quad , \text{where } \gamma(z_{nk}) = \frac{\pi_k \mathcal{N}(x_n|\mu_k,\Sigma_k)}{\sum_{j=1}^N \pi_j \mathcal{N}(x_n|\mu_j,\Sigma_j)}$$

(As shown in Bishop page 618)

I am trying to impose constraints on the observation model such that the latent states are ordered according to the mean. (i.e., $\mu_1 > \mu_2 > \mu_3 ...$). This will make the problem

$$argmax_{\mu_k} L \quad \text{such that } (\mu_1 - \mu_2) > 0, (\mu_2 - \mu_3 > 0), ... $$

How can I incorporate this constraint to get an expression for $\mu_k$?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.