2
$\begingroup$

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. I want my clustering procedure to understand that larger circles are more relevant than smaller ones without imposing any prior to the radius parameters or anything.

To me it seems reasonable to use a "weighted" Gaussian mixture model (GMM) with the following log-likelihood:

$$ L(x|\mu,\Sigma)= \sum_{i=1}^N R_i \log\big[ \sum_{j=1}^K\pi_j\mathcal{N}(x_i|\mu_j, \Sigma_j)\big] $$

where $x_i$ is the center of each circle, $\pi_j$ are the $K$ admixture parameters of the Gaussian densities $\mathcal{N}$ with mean $\mu_j$ and covariance matrix $\Sigma_j$. After training, I can then use the inferred admixture parameters to assign each circle to a cluster.

I am trying to recast the problem into the following: clustering integer radius circles seems to be equivalent to clustering degenerate points, i.e. data where the circles have been removed and replaced with points at the same locations $x_i$ in the plane and replicated "$R_i$ times". The advantage is that I can use the traditional (unweighted) GMM clustering (e.g. without modifying the ME algorithm) for the points with log-likelihood

$$ L(x|\mu,\Sigma)= \sum_{i=1}^{N^\prime} \log\big[ \sum_{j=1}^K\pi_j\mathcal{N}(x_i|\mu_j, \Sigma_j)\big] $$

where $N^\prime=\sum_i^N R_i$.

My questions are the following:

(1) I am far from being an expert in the subject and I would like to know if the two scenarios presented above, i.e. "weighted GMM" clustering for circles with different radii vs traditional GMM clustering for replicated points, are actually equivalent. (2) If yes, Will there be any pathologies when using GMM clustering on the replicated data? (3) Any advice on how to proceed with non-integer radii? Any comments would be very much appreciated.

$\endgroup$
2
  • 1
    $\begingroup$ You can trivially add point weighting to GMM, but that is not the same as a radius of a sphere around each point. $\endgroup$ Commented Feb 4, 2019 at 7:17
  • $\begingroup$ Thanks for the reply, the radius is just a way of visualizing the weight. How can I weigh the GMM? Any documentation that you can point me to? $\endgroup$
    – Dar Far
    Commented Feb 5, 2019 at 13:14

1 Answer 1

1
$\begingroup$

Adding weights to GMM is trivial.

No changes in the E step.

In the M step, after the E step, points are weighted in GMM based on the probability belonging to each cluster already. This probability is normalized to 1 because of the law if total probability (we assume every data point is to be fully explained by the model). If you multiply these weights with a point weight, you get GMM for weighted data. This is equivalent to repeating the point w times, or to normalizing the probabilities not to 1, but to w.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.