# Is it valid to reduce dimensionality of the data with PCA before running nonlinear dimensionality reduction?

I have some very high dimensional data, and performing Locally Linear Embedding (LLE) is very time consuming. I also have to perform several LLEs, with varying parameters, to compute the optimal number of neighbors via Spearman's Rho (a la Karbauskaite et al, see citation).

However, the data are highly linearly dependent--less than 10% of the features are necessary to preserve 98% of the variance. This does not preclude the data still being embedded in a lower dimensional manifold. Thus, my question is, is it valid to perform principal component analysis (PCA) to preserve the vast majority of the variance, followed by LLE, or will that radically change the results of the LLE? This would significantly reduce my compute time--so I'm hoping that can be done.

As an aside, the Spearman's Rho is computed based on pairwise distances--although this requires using the Geodesic distance across the manifold, NOT the Euclidean distances. Is there a way to compute Geodesic distances based on Scikit-Learn's implementation of PCA?

• I guess LLE means local liner embedding, but generally it's a good idea to decipher all abbreviations. Also, it would be helpful to give explicit links or citations if you mention a paper ("Karbauskaite et al"). – amoeba Sep 11 '14 at 22:27
• Thank you so much, and apologies for my poor composition! I'll add the citation. – eriophora Sep 12 '14 at 20:08

I am not an expert specifially on locally linear embedding, but I am sure that preprocessing with PCA would typically not affect the results of LLE (assuming that you keep enough components). Indeed, if the data are lying on a low-dimensional nonlinear manifold of dimensionality $d_1$, then this manifold can be itself contained in a linear manifold of dimensionality $d_2>d_1$ that is still smaller than the full dimensionality of that data $d_3>d_2>d_1$.