I try to do an (unsupervised) clustering with sklearn, Python, by different algorithms (hierarchical, distance based, density based etc. ones). The data in question has few hundred original features and I do experiments with different dimension reduction algorithms before the actual clustering (PCA, IncrementalPCA, LocallyLinearEmbedding, Isomap, FactorAnalysis).
The strange tendency I observe is that the fewer component I keep with dimension reductions (more-less irrespective to the kind of the algorithm) the better result I get (by silhouette score). This tendency is true to the extreme: the best result seems to be the one with only 2 compressed final dimension.
This seems absurd, my question is what is the explanation here? I could imagine that either silhouette score is misleading in the evaluation (but then what to chose?), or that I may still happen to have a few key original explanatory variable which is enough to be kept?