2
$\begingroup$

In online clustering there are approaches where a threshold $r$ on the distance to the nearest cluster is used to determine whether a new data point should be associated to an existing cluster or become its own cluster.
This kind of hyperparameter appears to me to be the kind that is somewhat difficult to tune, as it is not only dependant on the feature space itself but also the actual density of instances within it.

While the context of this question in particular is that I need to employ the approach by Souza et al. that I referenced below, which proposes a data stream classification model based on such a clustering method, this question doesn't doesn't need to be restricted to it:

Assuming there is a criterion $s(r)$ that assesses the quality of such a distance-threshold-based model, how could a set of threshold values to be evaluated be constructed given an observed data sample $X$ (and $\mathbf{y}$, in my case) ?

There is a question about a particular parameter choice for this type of approach, but I couldn't find any references to the presumed threshold value.

As is sadly often the case, the authors of the referenced article provide no information as to how they determined the used value for $r$ in their experiments.

Reference:

Souza, V. M., Silva, D. F., Batista, G. E., & Gama, J. (2015, December). Classification of evolving data streams with infinitely delayed labels. In Machine Learning and Applications (ICMLA), 2015 IEEE 14th International Conference on (pp. 214-219). IEEE.

$\endgroup$
  • 1
    $\begingroup$ Much of the stream clustering stuff seems to be impractical nonsense, yes. Never tested on real data, often even on stupid things like poker hands data artificially "streamed"... $\endgroup$ – Anony-Mousse Feb 16 at 21:56
  • $\begingroup$ To break a lance for the literature, there are sadly very few real-world data sets available for research purposes in this field. $\endgroup$ – deemel Feb 17 at 18:37
  • 2
    $\begingroup$ Which isn't really a good excuse for just making up data such as the poker hands "stream", isn't it? $\endgroup$ – Anony-Mousse Feb 18 at 0:55
0
$\begingroup$

In absence of another approach I opted for the following procedure, which might be helpful for others struggling with similar hyperparameters:

  1. Given a training set $X_{train}$ or (if computation power doesn't allow) a random sample from it, compute the pairwise distances matrix $D_X$.
  2. Evaluate the empirical CDF $\hat{F}(D_X)$.
  3. Define the lower and upper bounds of your parameter space for the threshold $r$ based on this.

I selected the minimum and maximum of the set of values for which $f=\hat{F}(d)$ satisfied both $d>0$ and $f<=0.4$, although one could of course extend this

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.