# Questions tagged [manifold-learning]

Manifold learning subsumes techniques conceived for problems where data of interest are assumed to lie on an embedded non-linear manifold within a higher-dimensional space.

89 questions
Filter by
Sorted by
Tagged with
10 views

### How to interpret Diffusion Maps for the iris dataset?

This might be a poor exercise but I'm trying to understand the methods of paper and if it makes sense to adapt my linear-based workflow with PCA to non-linear manifold methods; thought trying out ...
• 1,472
43 views

### Why do we expect the eigenvalues of the Gramian matrix found by Maximum Variance Unfolding to capture the number of degrees of freedom in the data?

Maximum Variance Unfolding (MVU) is a manifold learning method which, like other forms of dimensionality reduction, makes the assumption that whatever (high-dimensional) data we're dealing with "...
• 169
1 vote
80 views

### Is it correct to do SVD from the latent space of an autoencoder?

Is it correct to do SVD from the latent space of an autoencoder? I am asking because I think that by performing SVD from a latent space, and plotting the singular values, it is possible to know the ...
• 11
345 views

### Explanation of UMAP assumptions

Uniform Manifold Approximation (UMAP) is a technique for dimensionality reduction and visualization. The author of UMAP states that the algorithm is founded on three assumptions about the data: The ...
• 4,754
328 views

### What is the relationship between noise reduction and dimension reduction?

My understanding is that unsupervised methods like PCA, autoencoders and K-means shape a data space such that the modified representation of the data either nicely separates different families of data ...
104 views

### Determine if high dimensional data is multimodal

I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n. What are ...
• 21
111 views

### Why does data get so tangled up in high dimension?

When I look at textbooks on classification and machine learning, many of the examples focus on data that is often twisted up such as to avoid linear separation. I have an example picture below. The ...
• 1,522
204 views

### Reproducing kernel hilbert space norm as smoothness functional

Let $K:X \times X \rightarrow \mathbb{R}$ be a Mercer kernel with an associated RKHS $H$ then the norm $|f|_H^2$ can be used as a way to ensure that $f$ is smooth in $H$. If i understand correctly, ...
• 183
35 views

### Any theory on whether good choices of $k$ depend on $N$ and $D$ in KNN classification?

I am well aware that cross validation is a usual method for selecting hyperparameters. However, I am looking for theoretical guidance on how to pick $k$, the number of neighbors, for a $k$-nearest-...
• 741
1 vote
791 views

### Variance used in t-SNE

While the original paper of t-SNE is based on the SNE and SNE uses $\sigma_i^2$ (note the subscript $i$) while calculating the similarity of point $x_i$ WRT other points $j$. i.e., for each data point ...
1 vote
30 views

### Analytical tools to analyze the characteristics of a data manifold

In the paper "Emergence of separable manifolds in deep language representations," the authors use an analytical tool called Mean Field Theoretic Manifold Analysis to measure the manifold ...
• 11
1k views

### Dimension reduction using space filling curve to avoid "Curse of dimensionality"?

In machine learning, we want to train a model. While training, if the dimension of data is high, we have a problem (Curse of Dimensionality), so we want to reduce the dimension of our data. Since we ...
1 vote
761 views

### What validation if KFold scores differ a lot? Repeated KFold, LOO or Holdout?

Suppose you are given a medium-sized dataset and you did a KFold validation once. You notice that scores on each old differ noticeably. Which validation type is the most practical? I thought about ...
62 views

### State-of-the-art methods for out-of-sample-extension

I'm using a kernel based dimensionality reduction algorithms, and interested in extending out-of-sample data points for further analysis. I've been using the Nystrom method for this task, and some ...
• 839
32 views

### Simulate data that rotate on itself like DNA

I'm doing an exercise on the reduction of nonlinear dimension in manifold. I want to use LLE for that. And I'd like to simulate data in the form of DNA, so it rotates on itself. That is, a flat ...
• 121
1 vote
173 views

### how do i calculate (and Apply) Loss gradients with respect to the input (not the weights) of a CNN?

I have a trained generator, i would like to apply a loss function to the output and optimize the input (latent vector) using a gradient decent optimizer. i don't know how to calculate the gradients ...
• 11
1 vote
177 views

### Manifold Hypothesis vs. Latent Variables Assumption [closed]

As I understand: The manifold hypothesis claims that real world data, although represented in high dimension space, actually lies on a manifold in that space. I.e. that the actual data structure is of ...
• 3,520
1k views

### Multivariate Jensen-Shannon divergence

This paper says multivariate Jensen-Shannon divergence is $$JS(\mathbf{p}_1,\dots,\mathbf{p}_K) = \frac{1}{m} \sum KL(\mathbf{p}_i || \bar{\mathbf{p}})$$ with $KL$ being the KL-divergence of the ...
• 4,015
1 vote
717 views

### Differentiable PCA? [closed]

Is there a differentiable method for dimensionality reduction that is either based on PCA or has the properties of: Mathematically or algorithmically defined, e.g. not trained like an ML model or t-...
• 271
68 views

### PCA for non linear manifolds - Yann Lecun, Deep Learning Course, Question from a Lecture

I was watching the following lecture and at the very end of it, one of the students asked LeCun about using PCA for expression and pose feature extraction. https://www.youtube.com/watch?v=0bMe_vCZo30&...
1 vote
431 views

### Testing of hypothesis for the linearity of a data? PCA suggested, but how do we design a statistical test using it? [closed]

Suppose we're given the data set $\{x_1 \dots x_n\}$ in $\mathbb{R}^D$ the $D$-dimensional Euclidean space, and assume this data has intrinsic dimension $d < D.$ N.B. this just means that data is ...
• 761
1k views

### Meaning of "Manifold of interest"

While reading a paper, I stumble upon the following sentence and couldn't figure out its meaning: Informally, for an input set of real images, we say that the set of layer activations (for any ...
38 views

### Theoretical justification behind assuming that the data is locally uniformly distributed, as seem to be used by manifold learning community

In at least three or more papers I've been studying that introduced novel algorithms for the estimation of intrinsic dimensionality (ID) based on nearest neighborhood (NN) techniques, I observed that ...
• 761
1 vote
22 views

• 2,267
41 views

### How to identify manifolds for an optimisation problem

I don't have much experience in topology, but I am interested to know if: • Given a particular problem and associated cost function, how would one deduce what kind of manifold this problem lies on. ...
• 545
104 views

### Topological approach to create a space between clouds

I have a dataset associated with labels. According to https://arxiv.org/pdf/1802.03426.pdf --> UMAP (Uniform Manifold Approximation and Projection) which is a novel manifold learning technique for ...
• 11
48 views

• 253