I'm wondering if the following statement is correct:

After applying PCA distance between the same points (of course dimensions changed) have the same relation in comparison to distance between other points in relation to the original matrix.

In other words, if points have been close together in original matrix D, then the same points are also close together in D'=PCA(D). And also the reverse statement that remote points in D are also remote points in D', correct?

In addition I'm wondering if the following statement is correct:


$ p_a^n,p_b^n,p_c^n,p_d^n \in R^n$

$ p_a^k,p_b^k,p_c^k,p_d^k \in R^k $

where $dist$ is Euclidian distance.

In other words if distance between A and B divided by distance between C and D is X in D, is it also X in D'?

Thanks a lot!

EDIT 11. Nov. 16:

Although I've really enjoyed reading the links in the comments I think I have to express better what I want to know. I want to know if is correct to say that the proportion / ratios of euclidian distances in original data set $D$ stays thesame after transformig to $D'$ using $D'=pca(D)$

IMHO this should be the case since the principal components are calculated as follows:

$\mathbf{w}_{(1)} = \underset{\Vert \mathbf{w} \Vert = 1}{\operatorname{\arg\,max}}\,\left\{ \sum_i \left(t_1\right)^2_{(i)} \right\} = \underset{\Vert \mathbf{w} \Vert = 1}{\operatorname{\arg\,max}}\,\left\{ \sum_i \left(\mathbf{x}_{(i)} \cdot \mathbf{w} \right)^2 \right\}$

So basically estimating (optimizing on) $\mathbf{w}_{(1)}$ and performing a linear transformation (vector multiplication) which X IMHO the property above should be satisfied.

But if this is the case, how can I formally express this property using mathematical notation? My try you can see above, is this correct to say? I think I have to express it somehow this way, correct?


$ \forall p_a^n,p_b^n,p_c^n,p_d^n \in R^n$

$ \forall p_a^k,p_b^k,p_c^k,p_d^k \in R^k $

where dist(X,Y)="euclidian distance"

  • $\begingroup$ Just use $ twice and put inside the math text $\endgroup$ – Tommaso Guerrini Nov 10 '16 at 20:03
  • $\begingroup$ It looks like your question explores the theme how PCA is a form of MDS. $\endgroup$ – ttnphns Nov 10 '16 at 20:10
  • $\begingroup$ Although it's not really clear what you are asking, it seems to be whether PCA changes Mahalanobis distance. If that's the gist of it, then you should be able to easily deduce the answer from the description I gave at stats.stackexchange.com/questions/62092/… $\endgroup$ – whuber Nov 10 '16 at 21:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.