I'm working on a project involving PCA, and my knowledge up till now with this method is quite good. My work involves finding nearest neighbors (having the least Euclidean distance) to a particular spectrum in a database. So I reduce dimensionality of this database using PCA, where I project all the spectra onto the PCA space. Then I proceed to finding the spectrum's closest neighbors using the projected coefficients.
When I visualize PCA in a 2d space, I think of examples where a small distance in the PCA space do not correspond at all to a small distance in the original space. If the original space is 3d, and the PCA space is 2d for example, data-points lie "above" and "below" the PCA space(2d plane). So data-points might have similar projections, but in the original space are far from being close to each other. (Please correct me if I'm wrong)
My question is: is there a way to quantify this idea in order to achieve more accurate nearest neighbor search? And is there a way to represent the distance between the original and projected data-points in the PCA space (knowing that this distance is always orthogonal to the space)? P.S. I'm not a mathematician and I apologize for any incorrect terms.