1
$\begingroup$

So in classical fuzzy k-means clustering, the objective function is $\sum_i \sum_j u_{ij} \|x_i - c_j\|^2$

Now, we want to regularize this objective function using the entropy: $\sum_i^n H(U_i) = - \sum_i^n \sum_j^k u_{ij} \log u_{ij}$ where $u_{ij}$ is cluster membership for sample $x_i$ to $j$th cluster when we have total of $k$ clusters. So the objective function has is $$\sum_i \sum_j u_{ij} \|x_i - c_j\|^2 + \lambda \sum_i^n \sum_j^k u_{ij} \log u_{ij}$$

Using EM algorithm, we want to solve cluster memberships $u_{ij}$, by guessing cluster centers $c_j$ in the E-step, and then updating them in the M-step. Any idea how to solve $u_{ij}$ when $c_j$ are given?

$\endgroup$
1
$\begingroup$

I am not quite sure I see the problem. Please correct me if I am wrong but the objective function is separable (since there are no cross terms). Thus, you can for each (i,j) pair minimize $u_{ij} || x_{i} -c_{j}||^{2} + \lambda u_{ij} log(u_{ij})$ separately (in this step $c_{j}$ is known). Then rinse and repeat.

$\endgroup$
  • $\begingroup$ I guess the poster wants the additional constraints $\forall_i\sum_j u_{ij}=1$ $\endgroup$ – Anony-Mousse May 7 '15 at 6:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.