1
$\begingroup$

I want to use the Bayesian Information Criterium (BIC) to infer the number of clusters in a clustering algorithm, but I have two doubts.

Suppose I have $N$ points in a $D$-dimensional space, and that I am trying to fit $K$ clusters. I understand that the BIC is computed as $$ \mathit{BIC} = k \log n - 2 \log \hat L $$ where $k$ is the number of free parameters, $n$ is the "sample size", and $\hat L$ is the maximum value of the likelihood.

I am confused because, from what I understand (and see in various implementations of the BIC calculations, for example in the Python sklearn package), the sample size is just the number of datapoints, i.e. $n \equiv N$. However, naively, I would assume that the dimension of the data space $D$ must play a role, and I would rather set $n = N D$. Indeed, I can imagine situations where the $N$ multi-dimensional datapoints are used as $ND$ one-dimensional datapoints, and I see why the BIC should change as a consequence of this difference.

Secondly, in some clustering algorithms, I think it is possible to include some weights $w_n$ for each point. In my opinion it would be sensible to reformulate $n$ as $$ n = \frac{\bigl(\sum_{n=1}^N w_n \bigr)^2}{\sum_{n=1}^N w^2_n} \; , $$ or a similar equation in case where the dimension $D$ is to be taken into account. Is such a choice reasonable?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

The number of free parameters usually depends on the dimensionality.

If you have k' multivariate gaussian clusters, then k = k' * D²

For weighted, make sure that all weights equal should lead to the standard result. This clearly does not hold for your proposed equation.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I know $k$ might depend on the dimensionality, but not necessarily: imagine the case where I force all gaussian cluster to be "spherical", so $k = k'$. Also, if all weights are equal then $n = (N w)^2 / (N w^2) = N$, which is the standard result. $\endgroup$
    – Marco Lombardi
    May 2, 2018 at 9:16
  • $\begingroup$ No, for spherical you have to use e.g. k=k'*D because there are D free parameters in each Gaussian (actually probably even D+2) $\endgroup$
    – Has QUIT--Anony-Mousse
    May 2, 2018 at 18:12
  • $\begingroup$ And you equation yields n=N/N=1 if all w=1... $\endgroup$
    – Has QUIT--Anony-Mousse
    May 2, 2018 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.