2
$\begingroup$

I have a high dimensional data and when I applied UMAP on the whole data, it didn't seem to find a low dimensional manifold. Out of curiosity, I chose 3 important features and applied UMAP on that subset of the data. Here is result of applying UMAP on a data with 3 features. enter image description here

Then I just plotted the data in 3d to see how it differs from UMAP and here is the result:

enter image description here

My question is, what does UMAP do on a data with 3 dimensions? There is no manifold to find. Does it cluster the data? (I applied spectral clustering on Laplacian kernel of the data and the data points are colored based on the spectral clustering output).

$\endgroup$
5
  • $\begingroup$ Your plots make it look like you reduced down to three dimensions, yet you apply the UMAP on just three features. $\endgroup$
    – Dave
    Sep 23 at 0:21
  • $\begingroup$ @ Dave. That's what I did and my question is what is UMAP doing? $\endgroup$
    – mathlover
    Sep 23 at 0:25
  • $\begingroup$ Are you sure it does anything? $\endgroup$
    – Dave
    Sep 23 at 0:32
  • $\begingroup$ @Dave Compared to regular 3d plot, it shows some separated clusters $\endgroup$
    – mathlover
    Sep 23 at 0:33
  • 1
    $\begingroup$ Since the perspective changes, it is hard to judge the plots. $\endgroup$
    – Dave
    Sep 23 at 0:34

1 Answer 1

2
$\begingroup$

UMAP applies to 3D data the same algorithm as always, there is no special treatment for 3D data. UMAP is intended to be used for nonlinear dimensionality reduction, so applying it without dimensionality reduction ($\mathbb R^3\to\mathbb R^3$) is a little odd but probably instructive.

As explained here, UMAP creates a nerve for the original dataset and then tries to embed this nerve into the target space while preserving the nerve weights as well as possible. This is an optimization problem that is solved iteratively.

Now, clearly, a good preserving map from $\mathbb R^3\to\mathbb R^3$ would be the identity. But the assignment of a nerve to a dataset, controlled by various UMAP parameters like e.g. n_neighbors, which restricts the number of points considered locally, is not injective, and hence there are many different datasets that get assigned the same nerve by UMAP. So it is clear that nerve preserving doesn't mean dataset preserving even for $\mathbb R^3\to\mathbb R^3$.

Unfortunately, from the three-dimensional plots, it is not possible to judge how the result of UMAP has changed the data other than by a mere orthogonal transformation. But changes are usually due to

  • the loss of information created by the construction of the nerve as described above,
  • the above iterative optimization converging to a bad local minimum (rather unlikely in your situation),
  • UMAP parameters like e.g. metric if something other than euclidean is used,
  • UMAP parameters like min_dist, which makes the embedded points be at least min_dist apart.
$\endgroup$
3
  • $\begingroup$ Thanks. It does not seem to be orthogonal transform since the distances between data points are not preserved. $\endgroup$
    – mathlover
    Sep 23 at 16:54
  • $\begingroup$ How do you know? Did you measure it? I cannot tell from the 3D plot. In my experience, it is better to avoid 3D plots, especially if they are not interactive (but even then...). $\endgroup$
    – frank
    Sep 23 at 17:03
  • $\begingroup$ I did measure the distance. These were actually interactive plots but didn't figure out how to post them. $\endgroup$
    – mathlover
    Sep 23 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.