# Regularizing soft kmeans with entropy

So in classical fuzzy k-means clustering, the objective function is $\sum_i \sum_j u_{ij} \|x_i - c_j\|^2$

Now, we want to regularize this objective function using the entropy: $\sum_i^n H(U_i) = - \sum_i^n \sum_j^k u_{ij} \log u_{ij}$ where $u_{ij}$ is cluster membership for sample $x_i$ to $j$th cluster when we have total of $k$ clusters. So the objective function has is $$\sum_i \sum_j u_{ij} \|x_i - c_j\|^2 + \lambda \sum_i^n \sum_j^k u_{ij} \log u_{ij}$$

Using EM algorithm, we want to solve cluster memberships $u_{ij}$, by guessing cluster centers $c_j$ in the E-step, and then updating them in the M-step. Any idea how to solve $u_{ij}$ when $c_j$ are given?

I am not quite sure I see the problem. Please correct me if I am wrong but the objective function is separable (since there are no cross terms). Thus, you can for each (i,j) pair minimize $u_{ij} || x_{i} -c_{j}||^{2} + \lambda u_{ij} log(u_{ij})$ separately (in this step $c_{j}$ is known). Then rinse and repeat.
• I guess the poster wants the additional constraints $\forall_i\sum_j u_{ij}=1$ – Has QUIT--Anony-Mousse May 7 '15 at 6:27