Skip to main content
48 votes
Accepted

What is a manifold?

In non technical terms, a manifold is a continuous geometrical structure having finite dimension : a line, a curve, a plane, a surface, a sphere, a ball, a cylinder, a torus, a "blob"... something ...
Benoit Sanchez's user avatar
15 votes

What is a manifold?

A (topological) manifold is a space $M$ which is: (1) "locally" "equivalent" to $\mathbb{R}^n$ for some $n$. "Locally", the "equivalence" can be expressed via $n$ coordinate functions, $c_i: M \to ...
Chill2Macht's user avatar
  • 6,309
13 votes

Dimension reduction using space filling curve to avoid "Curse of dimensionality"?

I think your intuition is right; moving from $\mathbb{R}^n$ to an affine parameter along a space-filling curve will discard information about what points are close to one another in the high-...
Nobody's user avatar
  • 2,045
10 votes
Accepted

What is the value of linear dimensionality reduction in the presence of nonlinear alternatives?

I'll expand on some of the points mentioned in the comments, and add a few more. Computational complexity. PCA is more efficient in terms of both time and memory than more complicated nonlinear ...
user20160's user avatar
  • 32.8k
10 votes

What is a manifold?

In this context, the term manifold is accurate, but is unnecessarily highfalutin. Technically, a manifold is any space (set of points with a topology) that is sufficiently smooth and continuous (in a ...
David Wright's user avatar
  • 2,261
9 votes

Dimension reduction using space filling curve to avoid "Curse of dimensionality"?

The problem of the curse of dimensionality is caused by the amount of data needed, in the worst case, to adequately represent the underlying distribution goes up exponentially in the number of ...
Dikran Marsupial's user avatar
7 votes

How to know when to use linear dimensionality reduction vs non-linear dimensionality reduction?

One approach is to learn more about the structure of the data. Dimensionality reduction supposes that the data are distributed near a low dimensional manifold. If this is the case, one might choose ...
user20160's user avatar
  • 32.8k
7 votes

What is the manifold assumption in semi-supervised learning?

First, make sure that you understand what an embedding is. It's borrowed from mathematics. Roughly speaking, it is a mapping of the data into another space (often called embedding space or feature ...
Benjamin Crouzier's user avatar
6 votes

What is the difference between manifold learning and non-linear dimensionality reduction?

Non-linear dimensionality reduction occurs when method used for reduction assumes that manifold on which latent variables are lying is, well... non-linear. So for linear methods manifold is a n-...
Sengiley's user avatar
  • 376
5 votes

Graphical intuition of statistics on a manifold

A family of probability distributions can be analyzed as the points on a manifold with intrinsic coordinates corresponding to the parameters $(\Theta)$ of the distribution. The idea is to avoid a ...
Antoni Parellada's user avatar
4 votes

Resampling points in R^n so that kernel density is roughly uniform

Your conditions require resampling $x_i$ with probability inversely proportional to the original density estimates at $x_i.$ This is obvious: only such weights will produce new density estimates that ...
whuber's user avatar
  • 327k
4 votes
Accepted

Meaning of "Manifold of interest"

Real images (images from reality) with dimensions (H, W, C) make up only a tiny subset of all possible (H, W, C) tensors. In ...
jsaporta's user avatar
  • 213
3 votes
Accepted

Why do I get weird results when using high perpexity in t-SNE?

One cannot have perplexity values larger than sample size. [I don't have time right now, but I will try to provide a brief mathematical explanation of this later.] A popular t-SNE tutorial https://...
amoeba's user avatar
  • 106k
3 votes

Which dimensionality reduction schemes are bijective?

So in general your question is a bit weird, dimensionality reduction is related to projection which, as the name says, puts something onto something but many things can be put on the same thing. (...
tibL's user avatar
  • 296
3 votes
Accepted

What exactly is called "embedding" in dimensionality reduction?

Fortunately, I found on these slides an example: The mapping $x = W z$ defines an embedding of an $m$-dimensional manifold in $p$-dimensional space where $x$ is the original data. Moreover, now I ...
user_anon's user avatar
  • 1,047
3 votes
Accepted

Visualizing High Dimensional weight space for perceptrons

As you're likely aware, a vector in a $D$-dimensional space can be described by a direction and a magnitude. The direction requires $D-1$ values to describe, and the magnitude requires one value to ...
Bridgeburners's user avatar
3 votes

Manifold Hypothesis and PCA

Kernel PCA might be helpful. It performs linear PCA of the data projected in a reproducing kernel hilbert space where the data almost always lies on a linear manifold
kwala_96's user avatar
3 votes
Accepted

Is there an accepted method to determine an approximate dimension for manifold learning

I am not quite sure if I understood your confusion correctly, if you accept the embedding principle (i.e. the "manifold assumption") the only way you can "decide" your dimension is to construct a ...
Henry.L's user avatar
  • 2,480
3 votes
Accepted

Clustering and manifold learning

Probably the thinking is something along these lines... In manifold learning we have data in $R^n$, and we want to learn a lower dimensional manifold that the data is close to lying on. A set of ...
Matthew Drury's user avatar
3 votes

Comparing ISOMAP residual variance to PCA explained variance

The original Isomap paper defined "residual variance" as follows (reference 42): $$\text{residual variance} = 1 - R^2(\hat D_M, D_y)$$ where $R$ is the Pearson correlation coefficient over ...
gdkrmr's user avatar
  • 141
3 votes
Accepted

Why does data get so tangled up in high dimension?

Textbook examples are not meant to represent the likelihood of encountering particular situations. They are meant to cover a wide range, and give you the ability to recognize and solve certain special ...
cbeleites unhappy with SX's user avatar
2 votes

Learning vector embeddings from distances

The method you're looking for is called multidimensional scaling (MDS). MDS is a family of methods that attempt to embed a set of points into a vector space such that distances in the embedding space ...
user20160's user avatar
  • 32.8k
2 votes

Which dimensionality reduction schemes are bijective?

An autoencoder might be what you are after. The idea is to define two functions, an encoder $f_{\theta}: \mathbb{R}^N\to \mathbb{R}^n$ and a decoder $g_{\phi}: \mathbb{R}^n\to\mathbb{R}^N$, where $N$ ...
Simon Segert's user avatar
  • 2,054
2 votes

Which dimensionality reduction schemes are bijective?

If you think of dimensionality reduction only through linear transformations $T: V \rightarrow U$, say $V = \mathbb{R}^n$ and $U \subset V$, the transformation can be described by a $n \times n$ ...
Iron4dam's user avatar
2 votes

Nonlinear Dimensionality Reduction: geometric/topologic algorithms vs. autoencoders

Before I attempt to answer your question I want to create a stronger separation between the methods you are referring to. The first set of methods I believe you are referring to are neighborhood ...
Armen Aghajanyan's user avatar
2 votes

Should I center the data when performing Laplacian Eigenmap or any other manifold learning?

There are several manifold learning algorithms for which it doesn't make a difference. Laplacian Eigenmaps and Isomap only depend on the distances between the points, and translating these points ...
Jakub Bartczuk's user avatar
2 votes
Accepted

What is meant by: neighbor identities are preserved in t-SNE?

It is a slightly dense sentence indeed. It tries to combine two central notions, "preserving neighbourhood identity" and "extending to multiple different low-d images", in one go. The original paper ...
usεr11852's user avatar
  • 44.7k
2 votes
Accepted

Simulate data that rotate on itself like DNA

The following R Code should be easy to grasp and modify or be translated into another programming language: ...
Bernhard's user avatar
  • 8,475
2 votes

What validation if KFold scores differ a lot? Repeated KFold, LOO or Holdout?

I would generally avoid leave-one-out cross-validation for performance evaluation as it tends to have a high variance. I use it a lot for model selection (e.g. hyper-parameter tuning) because for ...
Dikran Marsupial's user avatar
2 votes

What validation if KFold scores differ a lot? Repeated KFold, LOO or Holdout?

I've just been learning about this myself so I'll share what I found. I believe it is Efron(1983) who established that LOOCV is 'nearly unbiased', but suffers from very high variance, especially with ...
N Blake's user avatar
  • 579

Only top scored, non community-wiki answers of a minimum length are eligible