The image below depicts the distance-distance plot for a (robust) PCA fit of a real data set.
The distance-distance plot is described in greater detail on page 30-31 of (1) or page 2--3 of (0). It is a diagnostic tool for PCA analysis (robust or otherwise).
For each observation in the sample, the PCA distance-distance plot depicts the normalized distances of that point on the fitted PCA subspace ("score distances") versus the "orthogonal distance" of that point to the PCA subspace. (The dotted red lines on this plot are the appropriate cutoff values for identifying outlying observations, though in this instance outlyingness is not the main object of the question.)
As you can see, for this particular dataset, a couple of points lie near a straight line on the SD/OD plot. I was wondering if this particular configuration had a geometrical interpretation in term of how these points look like in the original (data) space.
In the particular case where the rank of the data is 2 and the number of PCA component used to construct the SD distance is 1, points aligned the SD/OD plot are also lying on a line in the original (two dimensional) data space.
What I have problem with is what does that tell us about the geometry of those points in the case (as below) where the data has 30 variables and the SD distances are based on 10 components.
- (0) Hubert et al. 2005, ROBPCA: A New Approach to Robust Principal Component Analysis. ungated copy.
- (1) Valentin Todorov, Peter Filzmoser (2010). An Object-Oriented Framework for Robust Multivariate Analysis. JoSS Vol. 32, Issue 3. ungated copy.