Timeline for PCA on correlation or covariance: does PCA on correlation ever make sense?
Current License: CC BY-SA 3.0
18 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 1, 2013 at 15:19 | comment | added | ttnphns |
@Lucozade: Gosh! I can indeed retrieve the cov_PCs from the corr_PCs Please, go and show it as addendum in your answer. (Really, any one cov PC is the linear combination of the entire set of corr PCs, and vice versa. So what?)
|
|
| Jul 1, 2013 at 14:34 | comment | added | Lucozade | @ttnphns: ...and cannot be turned "back" into PCs on covariances...you can re-express them as linear combinations of the original variables. This seems contradictory to me. Can you clarify what you mean here? I have shown using my own data that I can indeed retrieve the cov_PCs from the corr_PCs (based on an independent check), which is of course no surprise because raw or centered and standardized data are linked via a linear transformation, hence their PCs are also linked deterministically. | |
| Jul 1, 2013 at 14:32 | comment | added | whuber♦ | "Geometrically [quite] obvious," maybe: but not true! The PCA for the correlation, when back-transformed to the original units, is not the PCA for the covariance. That would amount to a claim that orthogonal matrices commute with arbitrary diagonal matrices, which is easily refuted algebraically or by counterexample. | |
| Jul 1, 2013 at 14:21 | comment | added | Lucozade | @ttnphns: ...if you need to stay close to raw data...: to me, that seems always a necessity. Can you give an example where you have liberty to transform your data and have liberty not to back-transform your solution to the original data & units? This would be like using substiution method for solving an integral and not bother expressing the solution for the transformed variable back terms of the original variable of integration. | |
| Jul 1, 2013 at 14:08 | comment | added | Lucozade | @ttnphns: in my experience, one always starts from raw data as the starting point of a PCA analysis. Cases where your input data of interest (dimensioned or dimensionless) are already centered and sphered seem pathological. | |
| Jul 1, 2013 at 13:59 | comment | added | Lucozade | @whuber: *Underlying this answer is an implicit assumption that the units in which data are measured have an intrinsic meaning. *. This may be assumed by some, but not by me. Rather, it is the PCA solution that is intrinsicically linked to the units of the input data. If you take away the link by standardization, without re-expression in terms of the original units, what is the use and applicability of the solution. | |
| Jul 1, 2013 at 13:50 | comment | added | Lucozade | @whuber: the issue is that correlation PCA comes built-in, by definition, with standardization from x to z as a forward transformation of the original data. However, the reverse transformation that brings the PCs back to those of the pertinent x (and not z) is the vital (often) missing end operation for correlation PCA. If this operation is added, the correlation PCA reduces to the covariance PCA. Geometrically, in terms of stretching and unstretching the hyperplane of best fit, this necessity is quite obvious. One cannot claim to have solved the problem for z only if data was given for x. | |
| Jun 30, 2013 at 12:59 | comment | added | whuber♦ | @ttnphns I'll stick by the "merely," thanks. Whether or not the implications are "profound," the fact remains that standardization of a variable literally is an affine re-expression of its values: a change in its units of measure. The importance of this observation lies in its implications for some claims appearing in this thread, of which the most prominent is "covariance-based PCA is the only truly correct one." Any conception of correctness that ultimately depends on an essentially arbitrary aspect of the data--how we write them down--cannot be right. | |
| Jun 28, 2013 at 22:59 | comment | added | ttnphns |
@whuber: I can't help approving your wise remarks... with the exception of that hazy going from covariance to correlation are merely changes of units. It is "merely" this if all the variances are equal (and all variables are same units), else implications are more profound (e.g. see my answer).
|
|
| Jun 28, 2013 at 21:45 | comment | added | whuber♦ | Underlying this answer is an implicit assumption that the units in which data are measured have an intrinsic meaning. That is rarely the case: we may choose to measure length in Angstroms, parsecs, or anything else, and time in picoseconds or millennia, without altering the meaning of the data one iota. The changes made in going from covariance to correlation are merely changes of units (which, by the way, are particularly sensitive to outlying data). This suggests the issue is not covariance versus correlation, but rather to find fruitful ways to express the data for analysis. | |
| Jun 28, 2013 at 20:52 | comment | added | ttnphns |
(Cont.) And in fact, Several lines below Jolliffe speaks of yet another "form" of PCA - PCA on X'X matrix. This form is even "closer" to original data than cov-PCA because no centering of variables are being done. And the results are usually utterly different. You could also do PCA on cosines. People do PCA on all versions of the SSCP matrix, albeit covariances or correlations are used most often.
|
|
| Jun 28, 2013 at 20:43 | comment | added | ttnphns | (Cont.) Jolliffe's citation states, that PCs obtained on correlations will ever be themselves and cannot be turned "back" into PCs on covariances even though you can re-express them as linear combinations of the original variables. Thus, Jolliffe stresses the idea that PCA results are fully dependent on the type of pre-processing used and that there exist no "true", "genuine" or "universal" PCs... | |
| Jun 28, 2013 at 20:42 | comment | added | ttnphns |
It is so amusing that your own answer, which is in tune with everything people here were trying to convey to you, remains unsettled for you. You still argue There seems little point in PCA on correlations. Well, if you need to stay close to raw data ("physical data", as you strangely call it), you really shouldn't use correlations since they correspond to another ("distorted") data.
|
|
| Jun 28, 2013 at 20:17 | history | edited | Lucozade | CC BY-SA 3.0 |
deleted 86 characters in body
|
| Jun 28, 2013 at 20:10 | comment | added | Nick Cox | The missing reference here is Jolliffe, I.T. 2002. Principal component analysis. New York: Springer. [various misspellings of the author's name are common in citations] | |
| Jun 28, 2013 at 20:08 | history | edited | Nick Cox | CC BY-SA 3.0 |
added 1 characters in body
|
| Jun 28, 2013 at 19:34 | history | edited | Lucozade | CC BY-SA 3.0 |
added 13 characters in body
|
| Jun 28, 2013 at 19:29 | history | answered | Lucozade | CC BY-SA 3.0 |