What is a manifold?

Question

In dimensionality reduction technique such as Principal Component Analysis, LDA etc often the term manifold is used. What is a manifold in non-technical term? If a point $x$ belongs to a sphere whose dimension I want to reduce, and if there is a noise $y$ and $x$ and $y$ are uncorrelated, then the actual points $x$ would be far separated from each other due to the noise. Therefore, noise filtering would be required. So, dimension reduction would be performed on $z = x+y$. Therefore, over here does $x$ and $y$ belong to different manifolds?

I am working on point cloud data that is often used in robot vision; the point clouds are noisy due to noise in acquisition and I need to reduce the noise before dimension reduction. Otherwise, I will get incorrect dimension reduction. So, what is the manifold here and is noise a part of the same manifold to which $x$ belongs?

Answer 1

In non technical terms, a manifold is a continuous geometrical structure having finite dimension : a line, a curve, a plane, a surface, a sphere, a ball, a cylinder, a torus, a "blob"... something like this :

It is a generic term used by mathematicians to say "a curve" (dimension 1) or "surface" (dimension 2), or a 3D object (dimension 3)... for any possible finite dimension $n$. A one dimensional manifold is simply a curve (line, circle...). A two dimensional manifold is simply a surface (plane, sphere, torus, cylinder...). A three dimensional manifold is a "full object" (ball, full cube, the 3D space around us...).

A manifold is often described by an equation : the set of points $(x,y)$ such as $x^2+y^2=1$ is a one dimensional manifold (a circle).

A manifold has the same dimension everywhere. For example, if you append a line (dimension 1) to a sphere (dimension 2) then the resulting geometrical structure is not a manifold.

Unlike the more general notions of metric space or topological space also intended to describe our natural intuition of a continuous set of points, a manifold is intended to be something locally simple: like a finite dimension vector space : $\mathbb{R}^n$. This rules out abstract spaces (like infinite dimension spaces) that often fail to have a geometric concrete meaning.

Unlike a vector space, manifolds can have various shapes. Some manifolds can be easily visualized (sphere ,ball...), some are difficult to visualize, like the Klein bottle or the real projective plane.

In statistics, machine learning, or applied maths generally, the word "manifold" is often used to say "like a linear subspace" but possibly curved. Anytime you write a linear equation like : $3x+2y-4z=1$ you get a linear (affine) subspace (here a plane). Usually, when the equation is non linear like $x^2+2y^2+3z^2=7$, this is a manifold (here a stretched sphere).

For example the "manifold hypothesis" of ML says "high dimensional data are points in a low dimensional manifold with high dimensional noise added". You can imagine points of a 1D circle with some 2D noise added. While the points are not exactly on the circle, they satisfy statistically the equation $x^2+y^2=1$. The circle is the underlying manifold:

Answer 2

A (topological) manifold is a space $M$ which is:

(1) "locally" "equivalent" to $\mathbb{R}^n$ for some $n$.

"Locally", the "equivalence" can be expressed via $n$ coordinate functions, $c_i: M \to \mathbb{R}$, which together form a "structure-preserving" function, $c: M \to \mathbb{R}^n$, called a chart.

(2) can be realized in a "structure-preserving" way as a subset of $\mathbb{R}^N$ for some $N \ge n$. (1)(2)

Note that in order to make "structure" precise here, one needs to understand basic notions of topology (def.), which allows one to make precise notions of "local" behavior, and thus "locally" above. When I say "equivalent", I mean equivalent topological structure (homeomorphic), and when I say "structure-preserving" I mean the same thing (creates an equivalent topological structure).

Note also that in order to do calculus on manifolds, one needs an additional condition which doesn't follow from the above two conditions, which basically says something like "the charts are well-behaved enough to allow us to do calculus". These are the manifolds most often used in practice. Unlike general topological manifolds, in addition to calculus they also allow triangulations, which is very important in applications like yours involving point cloud data.

Note that not all people use the same definition for a (topological) manifold. Several authors will define it as satisfying only condition (1) above, not necessarily also (2). However, the definition which satisfies both (1) and (2) is much better behaved, therefore more useful for practitioners. One might expect intuitively that (1) implies (2), but it actually doesn't.

EDIT: If you are interested in learning about what precisely a "topology" is, the most important example of a topology to understand is the Euclidean topology of $\mathbb{R}^n$. This will be covered in-depth in any (good) introductory book about "real analysis".

Answer 3

In this context, the term manifold is accurate, but is unnecessarily highfalutin. Technically, a manifold is any space (set of points with a topology) that is sufficiently smooth and continuous (in a way that can, with some effort, be made mathematically well-defined).

Imagine the space of all possible values of your original factors. After a dimensional reduction technique, not all points in that space are attainable. Instead, only points on some embedded sub-space inside in that space will be attainable. That embedded sub-space happens to fulfill the mathematical definition of a manifold. For a linear dimensional reduction technique like PCA, that sub-space is just a linear sub-space (e.g. a hyper-plane), which is a relatively trivial manifold. But for non-linear dimensional reduction technique, that sub-space could be more complicated (e.g. a curved hyper-surface). For data analysis purposes, understanding that these are sub-spaces is much more important than any inference you would draw from knowing that they fulfill the definition of manifold.

Answer 4

As Bronstein and others have put it in Geometric deep learning: going beyond Euclidean data (Read the article here)

Roughly, a manifold is a space that is locally Euclidean. One of the simplest examples is a spherical surface modeling our planet: around a point, it seems to be planar, which has led generations of people to believe in the flatness of the Earth. Formally speaking, a (differentiable) d-dimensional manifold X is a topological space where each point x has a neighborhood that is topologically equivalent (homeomorphic) to a d-dimensional Euclidean space, called the tangent space.