Questions tagged [manifold-learning]

Manifold learning subsumes techniques conceived for problems where data of interest are assumed to lie on an embedded non-linear manifold within a higher-dimensional space.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

Dimensionality reduction based on target value

I have a dataset including 100k high dimensional data (e.g. houses in LA) (dim=100, e.g. house parameters like area, distance to downtown, etc.). Below is the 2-component PCA representation of the ...
0
votes
1answer
63 views

Spectral embedding: interpretation of new dimensions

I'm trying to gain an intuition for the 2nd dimension in the spectral embedding of an S-shaped dataset as in this example: The 1st dimension seems to neatly capture the local similarity between ...
0
votes
0answers
24 views

Manifold in the context of machine learning

In the Deep Learning book by Goodfellow et al. on page 158 it is stated: In the context of machine learning, we allow the dimensionality of the manifold to vary from one point to another. This ...
0
votes
1answer
27 views

Linear and Non-Linear dimensionality reduction with missing variables

I'm trying to compare the two types of reduction by applying it to a list of ingredients for about 1000 sponge cakes. The ingredient lists do however have miss certain ingredients out for some cakes, ...
1
vote
1answer
33 views

Dimensionality Reduction for Optimally Preserving KNN

Do any dimensionality reduction techniques find embeddings which optimally preserve the K-nearest neighbors of each point? If no algorithm provably does this, are there algorithms which heuristically ...
3
votes
1answer
35 views

Quantifying a manifold folding unto itself

I have a dataset of ~7k scattered points in 3D which represents a manifold that may or may not "fold unto itself". Here's an example where this does happen (look at the top-right yellow triangles): ...
2
votes
0answers
27 views

Learning Manifolds using Gradient Descent

I have a feedforward neural network $F(W): \mathbb R^d \rightarrow \mathbb R^k$ with $Relu$ activation, $m$ neurones per layer, $L$ layers and softmax on the output layer. $W$ denotes the weight ...
1
vote
1answer
18 views

Is it necessary to constrain the size of the neighborhood in LLE to be less than the space dimensionality?

The wikipedia entry on Locally Linear Embedding (LLE) says that LLE can be broken into stages, the first of which is to learn a barycentric linear model of the data with its $k$-nearest neighbors: $...
2
votes
0answers
19 views

How to identify manifolds for an optimisation problem

I don't have much experience in topology, but I am interested to know if: • Given a particular problem and associated cost function, how would one deduce what kind of manifold this problem lies on. ...
0
votes
0answers
85 views

Topological approach to create a space between clouds

I have a dataset associated with labels. According to https://arxiv.org/pdf/1802.03426.pdf --> UMAP (Uniform Manifold Approximation and Projection) which is a novel manifold learning technique for ...
1
vote
0answers
32 views

Is low rank finite-iteration manifold identification possible?

In sparse optimization, I am trying to solve the problem $$ \min_{x\in \mathbb R^{n}} \quad f(x) + \|x\|_1 $$ and at optimality, $x^*$ may be sparse. If I define the sparse manifold as $\mathcal M = ...
1
vote
1answer
60 views

Residual learning approach + manifold

From: Wikipedia on Residual learning in ANNs "The intuition on why this works is that the neural network collapses into fewer layers in the initial phase, which makes it easier to learn, and thus ...
2
votes
2answers
103 views

What exactly is called “embedding” in dimensionality reduction?

In the following slide I do not understand the definition of the term embedding. In the third paragraph, it says it is a mapping from low-dim. to high-dim, but in the last paragraph it suggests that ...
0
votes
0answers
27 views

isomap transform dimensionality

Hi i can successfully fit isomap on data and get relevant features in the reduced space. ...
1
vote
0answers
40 views

Is it possible to compute residual variances of Laplacian Eigenmaps, Diffusion Map and t-SNE to have a scree plots as in PCA?

I will like to do comparative analysis of PCA vs. Laplacian Eigenmaps, Diffusion Map, and <...
0
votes
0answers
127 views

connection between residual variance and explained variance PCA

The publication introducing Isomap compares PCA and Isomap by means of $$\text{residual variance} = 1 - R^2(\hat D_M, D_y)$$ where $R$ is the standard linear correlation coefficient over all ...
2
votes
1answer
53 views

What is meant by: neighbor identities are preserved in t-SNE?

From the article on Stochastic Neighbor Embedding (by Geoffrey Hinton and Sam Roweis): Stochastic Neighbor Embedding (SNE) tries to place the objects in a low-dimensional space so as to ...
2
votes
1answer
88 views

How to determine number of components to retain in manifold learning?

With PCA, one can use the explained variance ratio and keep the number of components that explain 95% of the dataset. How does one do the same for manifold learning methods for dimensionality ...
2
votes
1answer
535 views

t-SNE, PCA and another technique - which one? [closed]

I am comparing dimension reduction techniques and I am utilizing them for data visualizations onto a plane – projections in 2D space. The input into a projection/dimension reduction techniques is a ...
2
votes
1answer
384 views

Why do I get weird results when using high perpexity in t-SNE?

I played around with the t-SNE implementation in scikit-learn and found that increasing perplexity seemed to always result in a torus/circle. I couldn't find any ...
1
vote
0answers
98 views

how to check the stability of t-SNE?

I am looking some technique that help us to test the stability of t-SNE for different times running.
0
votes
0answers
28 views

why we need to select the smallest singular values in locally linear embedding (LLE)?

I'm learning about locally linear embedding. The cost function for finding embedded data is given by $\Phi(X) = \Sigma_{ij}M_{ij}(X_i.X_j^T$) Why we need to select the $2^{nd}$ to $(P+1)^{th}$ ...
5
votes
1answer
651 views

How to know when to use linear dimensionality reduction vs non-linear dimensionality reduction?

I am trying to decide whether to use linear dimensionality reduction methods (eg. PCA) vs. non-linear dimensionality reduction methods (eg. t-SNE) for my high-dimensional data set. However, I know ...
1
vote
0answers
153 views

Nonlinear dimensionality reduction in Java/on JVM

Is there a (decently) fast implementation of Manifold Learning algorithms on JVM? I tried Smile library but it takes dozen minutes to run its Isomap on my computer, while scikit-learn ...
3
votes
0answers
194 views

Topological data analysis and evaluating dimensionality reduction

I did an exploration some time ago on using TDA tools to see how topological features change after application of some nonlinear dimensionality reduction methods. For example I found out that, for ...
2
votes
1answer
43 views

Help in understanding a formula used in dimension estimation from nearest neighbor distances

In this paper, Abstract—The intrinsic dimensionality of a set of patterns is important in determining an appropriate number of features for representing the data and whether a reasonable two- or ...
0
votes
1answer
99 views

Diffusion Maps implementation

Suppose I have n data points lying in $\mathbf{R}^3$. Then, after defining my Gaussian diffusion kernel $k(x,y)$ and computing matrix $K$, I obtain $P$, whose entries $P_{ij}=p(x_i,y_i)$ is the ...
27
votes
3answers
6k views

What is a manifold?

In dimensionality reduction technique such as Principal Component Analysis, LDA etc often the term manifold is used. What is a manifold in non-technical term? If a point $x$ belongs to a sphere whose ...
1
vote
1answer
35 views

Should I center the data when performing Laplacian Eigenmap or any other manifold learning?

Suppose I have a high dimensional non-stationary non-linear time series, then is it advisable to center the data on the mean when performing laplacian eigenmap? I've heard somewhere that when ...
3
votes
0answers
101 views

Resampling points in R^n so that kernel density is roughly uniform

Let's say we have points $x_1,\ldots,x_n\in\mathbb{R}^N$ and let $X=\{x_1,\ldots,x_n\}$. I wish to produce a resampling $y_1,\ldots, y_m\in X$ (allowing repetitions) such that the new kernel density ...
0
votes
0answers
157 views

How to apply Laplacian Eigenmap on high-dimensional time series?

Suppose I have a time series of T observations for M features My question in application of Laplacian Eigenmap is when forming an adjacency matrix $A_{i,j}$, should I create an T by T matrix for $A_{...
2
votes
2answers
1k views

Visualizing High Dimensional weight space for perceptrons

I am watching the Neural Network videos by Prof. Geoff Hinton. In there he talks about a high dimensional Weight Space for perceptrons. In particular, I am ...
1
vote
0answers
47 views

Visualize comments

During my bachelor thesis I gathered a bunch of comments, labeled with a 0 for containing no hate and 1 for containing hate. The ...
0
votes
1answer
374 views

Manifold Hypothesis and PCA

I have a few gaps in my understanding: I know that PCA does dimensionality reduction. It does so by finding transformations such that the projection of the data points onto certain lines minimize the ...
0
votes
1answer
148 views

Can I use distance covariance for similarity measure in manifold learning?

In manifold learning such as Laplacian Eigenmap, a common method of obtaining the similarity matrix (that measures "affinity" or "connectivity") is to use Gaussian kernel in terms of the data points' ...
2
votes
1answer
38 views

Retrieve the corresponding element from another data set?

The problem is not necessarily related to NLP, just hopefully by putting it this way it will help illustrate the problem. I have the pronunciations of a set of words from different people and myself ...
5
votes
1answer
184 views

Is there an accepted method to determine an approximate dimension for manifold learning

Apologies for the rather vague title, I had difficulty explaining the question without making the title obnoxiously long. The manifold hypothesis suggests that natural data exists on or close to a ...
1
vote
0answers
25 views

Understanding shape of laplacian-embedded signal

I have a periodic signal for which I: a) extract main signal mode b) split that signal mode into windows of size 20 and create vectors of size 20 and c) do Laplacian embedding for this collection ...
1
vote
1answer
292 views

Clustering and manifold learning

I was at a group study recently, and I think one of the points made was that clustering is like 0-dimensional manifold-learning. Is this right? What is the reason behind it?
9
votes
1answer
790 views

What is the difference between manifold learning and non-linear dimensionality reduction?

What is the difference between manifold learning and non-linear dimensionality reduction? I have seen these two terms being used interchangeably. For example: http://www.cs.cornell.edu/~kilian/...
7
votes
1answer
696 views

Nonlinear Dimensionality Reduction: geometric/topologic algorithms vs. autoencoders

As I understand there are three main approaches to nonlinear dimensionality reduction: Manifold learning (geometric/topologic algorithms like ISOMAP, LLE, LTSA) Autoencoders things that do not fit ...
1
vote
0answers
17 views

Can the weight matrix for the K-NN graph be substituted for the full kernel matrix in Lap-RLS manifold regularization?

In this paper the algorithm on page 4 describes computing a weight matrix for a data adjacency graph, and a kernel matrix. I was wondering if it would make sense to substitute the weight matrix for ...
11
votes
2answers
832 views

Graphical intuition of statistics on a manifold

On this post, you can read the statement: Models are usually represented by points $\theta$ on a finite dimensional manifold. On Differential Geometry and Statistics by Michael K Murray and ...
0
votes
1answer
148 views

Which dimensionality reduction schemes are bijective?

I am trying to get a feel for which DR scheme is most suitable for my problem, with the stipulation that the scheme is bijectiv; i.e. there is a two-way map between the low-dimensional manifold and ...
6
votes
1answer
437 views

What is the value of linear dimensionality reduction in the presence of nonlinear alternatives?

From the results I've seen, manifold learning methods seem to generally outperform PCA for complicated, very high-dimensional datasets like images or videos. This makes sense to me, since nonlinear ...
3
votes
1answer
425 views

Learning vector embeddings from distances

So... I have a set of entities $\mathcal{E} = \{e_i \mid i \in [1,n]\}$, and I have a proper distance metric defined over $\mathcal{E}\times\mathcal{E}$, call it $d$, so the distance between $e_i$ ...
0
votes
0answers
139 views

How to tune hyperparameters for LLE?

I'm running LLE using Scikit-Learn (with the LocallyLinearEmbedding class), but there are a few hyperparameters and I would like to use grid search with cross-...
4
votes
0answers
217 views

Kernel PCA vs principal curve analysis

Both principal curve analysis and kernel PCA provide the ability to find nonlinear PCA. Kernel PCA does this by finding principal components in a higher dimensional space. Principal curve analysis is ...
2
votes
1answer
645 views

Manifold learning: does an embedding function need to be well behaving?

I am trying to learn about manifold learning techniques; a family of methods in machine learning. According to this idea, there is a low ($d$) dimensional, hidden space where the real data generation ...
5
votes
1answer
1k views

Two broad categories of dimensionality reduction

As a starter in dimensionality reduction (DR), I recently acquired the following understanding. There are two very broad categories of DR techniques: We can compute an analytic form of mapping from ...