# Questions tagged [manifold-learning]

Manifold learning subsumes techniques conceived for problems where data of interest are assumed to lie on an embedded non-linear manifold within a higher-dimensional space.

85 questions
Filter by
Sorted by
Tagged with
14 views

### Determine if high dimensional data is multimodal

I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n. What are ...
56 views

### Why does data get so tangled up in high dimension?

When I look at textbooks on classification and machine learning, many of the examples focus on data that is often twisted up such as to avoid linear separation. I have an example picture below. The ...
15 views

### Reproducing kernel hilbert space norm as smoothness functional

Let $K:X \times X \rightarrow \mathbb{R}$ be a Mercer kernel with an associated RKHS $H$ then the norm $|f|_H^2$ can be used as a way to ensure that $f$ is smooth in $H$. If i understand correctly, ...
33 views

### Any theory on whether good choices of $k$ depend on $N$ and $D$ in KNN classification?

I am well aware that cross validation is a usual method for selecting hyperparameters. However, I am looking for theoretical guidance on how to pick $k$, the number of neighbors, for a $k$-nearest-...
38 views

### Variance used in t-SNE

I was going through the t-SNE, but I am bit confused. While the original paper of t-SNE is based on the SNE and SNE uses $\sigma_i^2$ (note the subscript $i$) while calculating the similarity of point ...
1 vote
20 views

### Analytical tools to analyze the characteristics of a data manifold

In the paper "Emergence of separable manifolds in deep language representations," the authors use an analytical tool called Mean Field Theoretic Manifold Analysis to measure the manifold ...
1k views

### Dimension reduction using space filling curve to avoid "Curse of dimensionality"?

In machine learning, we want to train a model. While training, if the dimension of data is high, we have a problem (Curse of Dimensionality), so we want to reduce the dimension of our data. Since we ...
18 views

### spherical embedding of image data

Embedding positive data images onto a spherical manifold using the spherical embedding method doesn't guarantee the generation of positive directional quantities as the embedding positions are ...
1 vote
415 views

### What validation if KFold scores differ a lot? Repeated KFold, LOO or Holdout?

Suppose you are given a medium-sized dataset and you did a KFold validation once. You notice that scores on each old differ noticeably. Which validation type is the most practical? I thought about ...
26 views

### State-of-the-art methods for out-of-sample-extension

I'm using a kernel based dimensionality reduction algorithms, and interested in extending out-of-sample data points for further analysis. I've been using the Nystrom method for this task, and some ...
29 views

### Simulate data that rotate on itself like DNA

I'm doing an exercise on the reduction of nonlinear dimension in manifold. I want to use LLE for that. And I'd like to simulate data in the form of DNA, so it rotates on itself. That is, a flat ...
1 vote
72 views

### how do i calculate (and Apply) Loss gradients with respect to the input (not the weights) of a CNN?

I have a trained generator, i would like to apply a loss function to the output and optimize the input (latent vector) using a gradient decent optimizer. i don't know how to calculate the gradients ...
1 vote
78 views

### Manifold Hypothesis vs. Latent Variables Assumption [closed]

As I understand: The manifold hypothesis claims that real world data, although represented in high dimension space, actually lies on a manifold in that space. I.e. that the actual data structure is of ...
354 views

### Multivariate Jensen-Shannon divergence

This paper says multivariate Jensen-Shannon divergence is $$JS(\mathbf{p}_1,\dots,\mathbf{p}_K) = \frac{1}{m} \sum KL(\mathbf{p}_i || \bar{\mathbf{p}})$$ with $KL$ being the KL-divergence of the ...
382 views

### Differentiable PCA?

Is there a differentiable method for dimensionality reduction that is either based on PCA or has the properties of: Mathematically or algorithmically defined, e.g. not trained like an ML model or t-...
52 views

### PCA for non linear manifolds - Yann Lecun, Deep Learning Course, Question from a Lecture

I was watching the following lecture and at the very end of it, one of the students asked LeCun about using PCA for expression and pose feature extraction. https://www.youtube.com/watch?v=0bMe_vCZo30&...
1 vote
246 views

### Testing of hypothesis for the linearity of a data? PCA suggested, but how do we design a statistical test using it? [closed]

Suppose we're given the data set $\{x_1 \dots x_n\}$ in $\mathbb{R}^D$ the $D$-dimensional Euclidean space, and assume this data has intrinsic dimension $d < D.$ N.B. this just means that data is ...
846 views

### Meaning of "Manifold of interest"

While reading a paper, I stumble upon the following sentence and couldn't figure out its meaning: Informally, for an input set of real images, we say that the set of layer activations (for any ...
29 views

### Theoretical justification behind assuming that the data is locally uniformly distributed, as seem to be used by manifold learning community

In at least three or more papers I've been studying that introduced novel algorithms for the estimation of intrinsic dimensionality (ID) based on nearest neighborhood (NN) techniques, I observed that ...
1 vote
15 views

31 views

### How to identify manifolds for an optimisation problem

I don't have much experience in topology, but I am interested to know if: • Given a particular problem and associated cost function, how would one deduce what kind of manifold this problem lies on. ...