2
$\begingroup$

If I have a dataset with hundreds of samples and thousands of features, and t-SNE does a good job of separating classes compared to others classifiers, I don't understand why I can't rerun the algorithm with an additional test sample and predict its class based on KNN. Even if the model is non-parametric, can't the robustness of the method in generating coherent clusters time after time (different seeds or omitting some of the training samples) be used as an argument for using this method empirically?

$\endgroup$
8
  • $\begingroup$ Because it isn't strictly a supervised learning algorithm per-se, it is an embedding, i.e., projecting high dimensions into a restricted space. $\endgroup$ Apr 12, 2022 at 8:54
  • $\begingroup$ ok but if in practice it makes a good classifier (as assessed by leave-one-out cross-validation)? $\endgroup$
    – SebDL
    Apr 12, 2022 at 9:37
  • $\begingroup$ In practice there are no labels in the training data inherently, so learning theory, i.e., PAC or similar, supervised learning doesn't apply. $\endgroup$ Apr 12, 2022 at 10:15
  • 1
    $\begingroup$ Some related questions: (1) stats.stackexchange.com/questions/238538/… (2) stats.stackexchange.com/questions/398734/… $\endgroup$
    – Sycorax
    Apr 12, 2022 at 12:21
  • 1
    $\begingroup$ What do you do for out-of-sample data? $\endgroup$
    – Dave
    Apr 12, 2022 at 23:21

1 Answer 1

2
$\begingroup$

t-SNE gives no function for embedding out-of-sample data in the low-dimensional space. Consequently, all of the usual machine learning notions about out-of-sample performance are out.

If you use a different dimension reduction approach, such as UMAP or PCA, and then develop a functioning model based on that reduced dimensionality, that’s fine.

Addressing your exact question about KNN, if you use KNN after a dimension reduction, you’re still using KNN as the classifier, not an unsupervised method.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.