PCA is based on $L_2$ distance and is maximizing variance along the PC axes.

What if we try a different distance measure (something else than $L_2$)?
Do any methods corresponding to PCA but with different distance measures exist?
Can they be more useful than the vanilla PCA under some settings?

  • 1
    $\begingroup$ What exactly minimization or maximization is on you mind? Please note that classic PCA aims to maximize variance (SS of deviations from the origin which is usually the centroid) along the PC axes, not to minimize anything. It is strictly because the deal is about squared deviations it appears (by pythagorean theorem) that PCA simulateneously minimizes the squared deviations from the PC axes - of what you are speaking. If comes to deal with nonsquared deviations then the two tasks will be separate aims. $\endgroup$ – ttnphns Sep 2 '17 at 10:25
  • 1
    $\begingroup$ @ttnphns, thank you for an enlightening comment. I am not sure what exactly I want from the method, but I am curious if something similar exists. Probably I want dimensionality reduction a la PCA but where the distance function is different. But that could be achieved by first doing some feature scaling and then applying vanilla PCA on the scaled data, I guess. I don't really know what I want here, but I am curious about any related ideas and their use. Maybe you could guide me towards a more precise formulation such that the question would be answerable properly. $\endgroup$ – Richard Hardy Sep 2 '17 at 10:38
  • 1
    $\begingroup$ I'm thinking, too, about some pre-processing (and this is only immediate/intuitive, probably false idea). Here it is. Center the variables of X. Take sq. root of the values (their abs., then give them back their sign). Compute X'X/(n-1), that is, covariance matrix w/o new centering. Perform PCA. I don't know currently to what extent it will successfully maximize the Sum of abs. dev. along the PCs, but it might work. $\endgroup$ – ttnphns Sep 2 '17 at 10:47
  • $\begingroup$ Well it seems to me that mathematically PCA amounts to finding orthogonal basis with respect to an inner product. Inner products are quadratic forms and quadratic forms always have a basis for which it's diagonal. This is not my mathematical sweet spot, but I don't think you can produce distances compatible with the vector space structure except by L2. $\endgroup$ – meh Jun 26 '18 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.