Say that we got a set $\mathcal{X} = \{x_1, x_2, \ldots\}$ of samples. You want to partition $\mathcal{X}$ into $k$ subsets, where $k$ is unknown besides the fact that $k \ge 1$. How clustering problem can be forumlated in probabilistic terms (as opposed to heuristics using distance measures)?

Here is an example: let $x$ be a sample, then for any cluster $i \in \{1, 2, \ldots, k\}$ find the probability $\Pr(C=i | X=x)$, where $C$ and $X$ are random variables that take values in sets $\{1, 2, \ldots, k\}$ and $\mathcal{X}$, respectively. How can you move forward with this approach? We don't even know $k$...


closed as unclear what you're asking by Glen_b Jan 30 '16 at 9:40

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 3
    $\begingroup$ Probability of what? There are probabilistic Finite Mixture Models (stats.stackexchange.com/questions/130805/…) but I am not sure if this is what you are asking. You can check also stats.stackexchange.com/questions/23472/… $\endgroup$ – Tim Jan 30 '16 at 8:08
  • 1
    $\begingroup$ @caveman but this is your question, so you have to formulate it to be answerable... $\endgroup$ – Tim Jan 30 '16 at 16:22
  • 1
    $\begingroup$ If you define it that broad see the first link that I posted that defines finite mixture model. Distribution of $X$ is defined as mixture of $k$ distributions such that: $f(x, \vartheta) = \sum_k \pi_k f_k(x, \vartheta_k)$ with parameters $\pi_k, \vartheta_k$, the parameters and $k$ itself have to be found. $\endgroup$ – Tim Jan 30 '16 at 16:39
  • 1
    $\begingroup$ @caveman I may provide such answer, but you have to edit your question to make it more precise and clear. As for now, as question is unclear, it is put on hold and nobody can answer it. We want this site to be knowledge repository, so we are looking for high-quality questions and answers that will be helpful not only for those who ask, but also for others. Editing to add information on probability of what you are interested in this problem would make this question more clear. $\endgroup$ – Tim Jan 31 '16 at 8:49
  • 1
    $\begingroup$ I edited it already hours after it was put on hold (thus expanded question section with example) and I don't see how more precision can be added there because the question itself is very broad. Maybe I am too stupid to see what I must do in order to comply. Anyway sucks to be me I guess. Thank you and highly appreciate your suppport $\endgroup$ – caveman Jan 31 '16 at 18:14

There are

  • Gaussian Mixture Modeling
  • Akaike Information Criterion
  • Bayesian Information Criterion

that are somewhat based on probabilistic considerations.

But in the end you use clutering if you do not know how to model your data. I.e. when you have no idea of the probabilities.

  • $\begingroup$ Am I right to think that mixture models (without adding Gaussian, cause it could be a mixture of non-Gaussians) is the most comprehensive answer? $\endgroup$ – caveman Jun 3 '16 at 0:24
  • $\begingroup$ Also may I know what have you suggested AIC and BIC? These seem to be heuristics to predict which model is better than another in the absence of ground truth information. Some kind of intrinsic evaluation criteria. Am I right? $\endgroup$ – caveman Jun 3 '16 at 0:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.