Observations aligned on PCA diagnostic plot: geometric interpretation?

The image below depicts the distance-distance plot for a (robust) PCA fit of a real data set.

The distance-distance plot is described in greater detail on page 30-31 of (1) or page 2--3 of (0). It is a diagnostic tool for PCA analysis (robust or otherwise).

For each observation in the sample, the PCA distance-distance plot depicts the normalized distances of that point on the fitted PCA subspace ("score distances") versus the "orthogonal distance" of that point to the PCA subspace. (The dotted red lines on this plot are the appropriate cutoff values for identifying outlying observations, though in this instance outlyingness is not the main object of the question.)

As you can see, for this particular dataset, a couple of points lie near a straight line on the SD/OD plot. I was wondering if this particular configuration had a geometrical interpretation in term of how these points look like in the original (data) space.

In the particular case where the rank of the data is 2 and the number of PCA component used to construct the SD distance is 1, points aligned the SD/OD plot are also lying on a line in the original (two dimensional) data space.

What I have problem with is what does that tell us about the geometry of those points in the case (as below) where the data has 30 variables and the SD distances are based on 10 components.

• (0) Hubert et al. 2005, ROBPCA: A New Approach to Robust Principal Component Analysis. ungated copy.
• (1) Valentin Todorov, Peter Filzmoser (2010). An Object-Oriented Framework for Robust Multivariate Analysis. JoSS Vol. 32, Issue 3. ungated copy.

• I've never seen such a diagnostic plot before. Apparently it comes from this paper: Hubert et al. 2005, ROBPCA: A New Approach to Robust Principal Component Analysis, see pp. 2-3. It looks like your diagonal line is due to some curious artifact: a bunch of points that are located on a line orthogonal to the "robust" 10-dim subspace. There is almost no "robust" variation in this 1-dim line, only these artifacts (that seem to be spaced in a particular pattern). They get pretty far from the "main cloud". – amoeba Oct 18 '16 at 22:05
• @amobea: thanks (also for the edit. I hope I didn't accidentally overwrite anything you changed, it seems we were editing the question simultaneously at some point). You mean as the point labeled 3 (or 2) on page 2 of that paper? The subspace orthogonal to the fitted PCA model seems the correct way to 'visualize' the geometry of these points. What I still not sure about is why you wrote a 'line'. Could it be that they lie on, say, a plane orthogonal to the PCA fit? Thanks for sharing your thoughts. – user603 Oct 19 '16 at 7:50
• Actually I was more thinking like point labeled 5... In general I am sure there can be different arrangements leading to the same diagnostic plot because a lot of information has been discarded for it. But one "line" with regularly spaced points somehow seems more probable to me than a plane or some other higher-dimensional arrangement producing such a regular pattern. – amoeba Oct 19 '16 at 9:38
• @amoeba: I think points placed as '5' would have low SD distances which these ones clearly don't~ – user603 Oct 19 '16 at 9:50
• Hmm, yes, that's probably true. – amoeba Oct 19 '16 at 9:56