Which dimensionality reduction schemes are bijective? I am trying to get a feel for which DR scheme is most suitable for my problem, with the stipulation that the scheme is bijectiv; i.e. there is a two-way map between the low-dimensional manifold and the original space.
EDIT:
For clarity: is it possible to select your dimensional reduction so that you can project UP from the low dimensional space that you have found, such that information loss is acceptable. E.g. say I have a high-dimensional joint configuration of some character, and I want to find its low-dimensional joint projection and then traverse that manifold, or sample it, and visualise those samples in the original space. 
 A: So in general your question is a bit weird, dimensionality reduction is related to projection which, as the name says, puts something onto something but many things can be put on the same thing. (Think about a point in 3D that is projected on a surface then think about the perpendicular line to the surface passing through that point. every point on that line is projected onto the same point)
Ok let's consider PCA which you claim is bijective (it really isn't), actually consider SVD since PCA is really just SVD. 
Full SVD is:
$ M = U\Lambda V^T$
when you consider dimensionality reduction, you only consider the first largest elements of the $\Lambda$, a diagonal matrix, hence the representation 
$\hat M = U\hat\Lambda V^T$
now this representation can correspond to an infinite number of matrices $M$. In other words, you can build an infinite number of matrices $M'$ such that, when applying the same process of SVD truncation, you get $\hat M$ (so it's really not bijective)
Indeed: let
$M' = U \Lambda' V^T$
where $\Lambda'$ has the same first few singular values than $\Lambda$ and all the others are different (but smaller). The SVD truncation of all these $M'$ will give you $\hat M$.
A: If you think of dimensionality reduction only through linear transformations $T: V \rightarrow U$, say $V = \mathbb{R}^n$ and $U \subset V$, the transformation can be described by a $n \times n$ matrix but it is not full rank as $\text{dim}(\text{im}(T)) = \text{dim}(U) < \text{dim}(V)$. The matrix is not invertible and there is no one-to-one mapping. Also by rank nullity theorem, $\text{dim}(\text{ker}(T)) = \text{dim}(V) - \text{dim}(\text{im}(T)) \geq 1$. So there is at least one line in the space that is mapped to 0 with this dimensionality reducing transformation, and the map is clearly not bijective.
A: An autoencoder might be what you are after. The idea is to define two functions, an encoder $f_{\theta}: \mathbb{R}^N\to \mathbb{R}^n$ and a decoder $g_{\phi}: \mathbb{R}^n\to\mathbb{R}^N$, where $N$ is the dimensionality of your data and $n<<N$. It is common to parametrize $f$ and $g$ by neural networks. Then each datum $X$ can be approximately reconstructed by $g_{\phi}(f_{\theta}(X))$, and the parameters $\phi,\theta$ are trained so as to minimize the average reconstruction loss.
Given any point in the latent space $\mathbb{R}^n$, you can simply run it throug the decoder $g_{\phi}$ to get back a point in the original space.
