Is it valid to reduce dimensionality of the data with PCA before running nonlinear dimensionality reduction?

Question

I have some very high dimensional data, and performing Locally Linear Embedding (LLE) is very time consuming. I also have to perform several LLEs, with varying parameters, to compute the optimal number of neighbors via Spearman's Rho (a la Karbauskaite et al, see citation).

However, the data are highly linearly dependent--less than 10% of the features are necessary to preserve 98% of the variance. This does not preclude the data still being embedded in a lower dimensional manifold. Thus, my question is, is it valid to perform principal component analysis (PCA) to preserve the vast majority of the variance, followed by LLE, or will that radically change the results of the LLE? This would significantly reduce my compute time--so I'm hoping that can be done.

As an aside, the Spearman's Rho is computed based on pairwise distances--although this requires using the Geodesic distance across the manifold, NOT the Euclidean distances. Is there a way to compute Geodesic distances based on Scikit-Learn's implementation of PCA?

Citation: Karbauskaitė, Rasa, and O. Kurasova G. Dzemyda. "Selection of the number of neighbours of each data point for the locally linear embedding algorithm." Information Technology and Control 36.4 (2007): 359-364.

Answer 1

I am not an expert specifially on locally linear embedding, but I am sure that preprocessing with PCA would typically not affect the results of LLE (assuming that you keep enough components). Indeed, if the data are lying on a low-dimensional nonlinear manifold of dimensionality $d_1$, then this manifold can be itself contained in a linear manifold of dimensionality $d_2>d_1$ that is still smaller than the full dimensionality of that data $d_3>d_2>d_1$.

According to the book "Nonlinear Dimensionality Reduction" by Lee and Verleysen, this is a useful approach, see sections 7.2.3-7.2.4:

When data dimensionality is very high, linear dimensionality reduction by PCA may be very useful to suppress a large number of useless dimensions. [...] This also eases the work to be achieved by subsequent nonlinear methods. [...] Nonlinear methods of dimensionality reduction may take over from PCA once the dimensionality is no longer too high, between a few tens and a few hundreds, depending on the chosen method.

Your second question I don't quite understand, but there is no way to compute geodesic distances by PCA. This can be done by or after the LLE step. Preprocessing with PCA just makes sure that your linear distances do not change much after the first (linear) dimensionality reduction step. If the whole nonlinear manifold is preserved, then the geodesic distances will not change either.