# Questions tagged [manifold-learning]

Manifold learning subsumes techniques conceived for problems where data of interest are assumed to lie on an embedded non-linear manifold within a higher-dimensional space.

89 questions
Filter by
Sorted by
Tagged with
16 views

### Why do we expect the eigenvalues of the Gramian matrix found by Maximum Variance Unfolding to capture the number of degrees of freedom in the data?

Maximum Variance Unfolding (MVU) is a manifold learning method which, like other forms of dimensionality reduction, makes the assumption that whatever (high-dimensional) data we're dealing with "...
36 views

### Is it correct to do SVD from the latent space of an autoencoder?

Is it correct to do SVD from the latent space of an autoencoder? I am asking because I think that by performing SVD from a latent space, and plotting the singular values, it is possible to know the ...
26 views

### Applying non-linear dimensionality reduction on binary data

Would be beneficial to apply nonlinear dimensionality reduction on a binary dataset (around 200 binary features) ? what is difference from applying MCA (multiple correspondence analysis).
195 views

### Explanation of UMAP assumptions

Uniform Manifold Approximation (UMAP) is a technique for dimensionality reduction and visualization. The author of UMAP states that the algorithm is founded on three assumptions about the data: The ...
161 views

### What is the relationship between noise reduction and dimension reduction?

My understanding is that unsupervised methods like PCA, autoencoders and K-means shape a data space such that the modified representation of the data either nicely separates different families of data ...
72 views

### Determine if high dimensional data is multimodal

I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n. What are ...
82 views

### Why does data get so tangled up in high dimension?

When I look at textbooks on classification and machine learning, many of the examples focus on data that is often twisted up such as to avoid linear separation. I have an example picture below. The ...
137 views

### Reproducing kernel hilbert space norm as smoothness functional

Let $K:X \times X \rightarrow \mathbb{R}$ be a Mercer kernel with an associated RKHS $H$ then the norm $|f|_H^2$ can be used as a way to ensure that $f$ is smooth in $H$. If i understand correctly, ...
33 views

### Any theory on whether good choices of $k$ depend on $N$ and $D$ in KNN classification?

I am well aware that cross validation is a usual method for selecting hyperparameters. However, I am looking for theoretical guidance on how to pick $k$, the number of neighbors, for a $k$-nearest-...
496 views

### Variance used in t-SNE

I was going through the t-SNE, but I am bit confused. While the original paper of t-SNE is based on the SNE and SNE uses $\sigma_i^2$ (note the subscript $i$) while calculating the similarity of point ...
1 vote
29 views

### Analytical tools to analyze the characteristics of a data manifold

In the paper "Emergence of separable manifolds in deep language representations," the authors use an analytical tool called Mean Field Theoretic Manifold Analysis to measure the manifold ...
1k views

### Dimension reduction using space filling curve to avoid "Curse of dimensionality"?

In machine learning, we want to train a model. While training, if the dimension of data is high, we have a problem (Curse of Dimensionality), so we want to reduce the dimension of our data. Since we ...
1 vote
636 views

### What validation if KFold scores differ a lot? Repeated KFold, LOO or Holdout?

Suppose you are given a medium-sized dataset and you did a KFold validation once. You notice that scores on each old differ noticeably. Which validation type is the most practical? I thought about ...
42 views

### State-of-the-art methods for out-of-sample-extension

I'm using a kernel based dimensionality reduction algorithms, and interested in extending out-of-sample data points for further analysis. I've been using the Nystrom method for this task, and some ...
31 views

### Simulate data that rotate on itself like DNA

I'm doing an exercise on the reduction of nonlinear dimension in manifold. I want to use LLE for that. And I'd like to simulate data in the form of DNA, so it rotates on itself. That is, a flat ...
1 vote
170 views

### how do i calculate (and Apply) Loss gradients with respect to the input (not the weights) of a CNN?

I have a trained generator, i would like to apply a loss function to the output and optimize the input (latent vector) using a gradient decent optimizer. i don't know how to calculate the gradients ...
1 vote
149 views

### Manifold Hypothesis vs. Latent Variables Assumption [closed]

As I understand: The manifold hypothesis claims that real world data, although represented in high dimension space, actually lies on a manifold in that space. I.e. that the actual data structure is of ...
908 views

### Multivariate Jensen-Shannon divergence

This paper says multivariate Jensen-Shannon divergence is $$JS(\mathbf{p}_1,\dots,\mathbf{p}_K) = \frac{1}{m} \sum KL(\mathbf{p}_i || \bar{\mathbf{p}})$$ with $KL$ being the KL-divergence of the ...
1 vote
602 views

### Differentiable PCA? [closed]

Is there a differentiable method for dimensionality reduction that is either based on PCA or has the properties of: Mathematically or algorithmically defined, e.g. not trained like an ML model or t-...
61 views

### PCA for non linear manifolds - Yann Lecun, Deep Learning Course, Question from a Lecture

I was watching the following lecture and at the very end of it, one of the students asked LeCun about using PCA for expression and pose feature extraction. https://www.youtube.com/watch?v=0bMe_vCZo30&...
1 vote
394 views

### Testing of hypothesis for the linearity of a data? PCA suggested, but how do we design a statistical test using it? [closed]

Suppose we're given the data set $\{x_1 \dots x_n\}$ in $\mathbb{R}^D$ the $D$-dimensional Euclidean space, and assume this data has intrinsic dimension $d < D.$ N.B. this just means that data is ...
1k views

### Meaning of "Manifold of interest"

While reading a paper, I stumble upon the following sentence and couldn't figure out its meaning: Informally, for an input set of real images, we say that the set of layer activations (for any ...
37 views

### Theoretical justification behind assuming that the data is locally uniformly distributed, as seem to be used by manifold learning community

In at least three or more papers I've been studying that introduced novel algorithms for the estimation of intrinsic dimensionality (ID) based on nearest neighborhood (NN) techniques, I observed that ...
1 vote
20 views

41 views

### How to identify manifolds for an optimisation problem

I don't have much experience in topology, but I am interested to know if: • Given a particular problem and associated cost function, how would one deduce what kind of manifold this problem lies on. ...
104 views

### Topological approach to create a space between clouds

I have a dataset associated with labels. According to https://arxiv.org/pdf/1802.03426.pdf --> UMAP (Uniform Manifold Approximation and Projection) which is a novel manifold learning technique for ...