Projecting data on a sphere I am used to working with PCA, tSNE, LLEs... They all do a great job projecting the data on a plane (or on linear subspaces of $\mathbb{R}^n$). Is there any other embedding technique that projects the data on non-linear spaces ? Like a sphere, or any other manifold ? 
The aim would be to specify a space and a distance (the example will be a sphere and the geodesic distance) to project onto. 
I am looking for any reference or open source project! 
Per example, calling $x_i$ the elements of the data set and $d$ the initial distance, $d_g$ the geodesic distance on the sphere, an "MDS-like" formulation could be :
$$ min_{y} \sum_{i,j}||d(x_i,x_j)-d_g(y_i,y_j)||^2 $$
Is there any other way than brute force to solve this problem ?
 A: It might be possible to solve your example problem using a procedure similar to nonclassical metric MDS (using the stress criterion). Initialize the 'projected points' to lie on a sphere (more on this later). Then, use an optimization solver to find the projected points that minimize the objective function. There are a few differences compared to ordinary MDS. 1) Geodesic distances must be computed between projected points, rather than Euclidean distances. This is easy when the manifold is a sphere. 2) The optimization must obey the constraint that the projected points lie on a sphere.
Fortunately, there are existing, open source solvers for performing optimization where the parameters are constrained to lie on a particular manifold. Since the parameters here are the projected points themselves, this is exactly what's needed. Manopt is a package for Matlab, and Pymanopt is for Python. They include spheres as one of the supported manifolds, and others are available.
The quality of the final result will depend on the initialization. This is also the case for ordinary, nonclassical MDS, where a good initial configuration is often obtained using classical MDS (which can be solved efficiently as an eigenvalue problem). For 'spherical MDS', you could take the following approach for initialization. Perform ordinary MDS, isomap, or some other nonlinear dimensionality reduction technique to obtain coordinates in a Euclidean space. Then, map the resulting points onto the surface of a sphere using a suitable projection. For example, to project onto a 3-sphere, first perform ordinary dimensionality reduction to 2d. Map the resulting points onto a 3-sphere using something like a stereographic projection. If the original data lies on some manifold that's topologically equivalent to a sphere, then it might be more appropriate to perform initial dimensionality reduction to 3d (or do nothing if they're already in 3d), then normalize the vectors to pull them onto a sphere. Finally, run the optimization. As with ordinary, nonclassical MDS, multiple runs can be performed using different initial conditions, then the best result selected.
It should be possible to generalize to other manifolds, and to other objective functions. For example, we could imagine converting the objective functions of other nonlinear dimensionality reduction algorithms to work on spheres or other manifolds.
A: Maybe something like that on R ?
http://planspace.org/2013/02/03/pca-3d-visualization-and-clustering-in-r/
The PCA project on a plane if you only take the first two axes. If you take the 3rd one you have the projection on sphere.
A: Relational Perspective Map (http://www.visumap.net/index.aspx?p=Resources/RpmOverview) might be something interesting for you. RPM has been originally purposed for torus surface; then extended to other low dimensional manifolds like 3D sphere and projective plane. A key to design MDS on closed manifold is that the "distance" metric has to be extended to a kind of multi-path-aggregation. The VisuMap software package also provides animation/nevigation to visualize data in spaces like 3D sphere.
