I've been reading a lot about $t$-sne algorithm for dimensionality reduction. I'm very impressed with the performance on "classic" datasets, like MNIST, where it achieves a clear separation of the digits (see original article):
I've also used it to visualize the features learnt by a neural network I'm training and I was very pleased with the results.
So, as I understand it:
$t$-sne has good results on most datasets, and has a pretty efficient implementation - $O(n \log n)$ with the Barnes-Hut approximation method. Then, could we potentially say that the "dimensionality reduction" problem, at least for the purpose of creating good 2D/3D visualizations, is now a "closed" problem?
I'm aware that this is a pretty bold statement. I'm interested in understanding what the potential "pitfalls" of this method are. That is, are there any cases in which we know that it is not useful? Moreover, what are the "open" problems in this field?