Timeline for When can I use $XX^\top/n$ as covariance matrix for PCA?
Current License: CC BY-SA 3.0
28 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 13, 2019 at 11:28 | vote | accept | LeoW. | ||
| May 12, 2019 at 12:31 | answer | added | kjetil b halvorsen♦ | timeline score: 2 | |
| May 12, 2019 at 12:31 | history | edited | kjetil b halvorsen♦ |
edited tags
|
|
| Apr 13, 2017 at 12:44 | history | edited | CommunityBot |
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
|
|
| May 11, 2016 at 16:25 | comment | added | Cagdas Ozgenc | Let us continue this discussion in chat. | |
| May 11, 2016 at 16:08 | comment | added | amoeba | @Cagdas You mean that sample covariance matrix will be a bad estimator of the population covariance matrix? Can be, yes. But sample covariance matrix is just XX'/n by definition. | |
| May 11, 2016 at 14:49 | comment | added | Cagdas Ozgenc | @amoeba Actually vice versa. There are restrictions to use XX'/n as a covariance matrix, those that I mentioned. | |
| May 11, 2016 at 14:08 | comment | added | amoeba | @CagdasOzgenc: That is fair enough. I think this would be an answer to the question "are there any restrictions to using sample covariance matrix in PCA?". But the answer to the question "are there any restrictions to using XX'/n with centered X as sample covariance matrix?" is simply "no". In any case, I encourage you to post an answer here elaborating on all that. | |
| May 11, 2016 at 14:06 | comment | added | Cagdas Ozgenc | @amoeba He asked if there are any restrictions. I would say if the distribution is fat tailed (or in extreme case like Cauchy) then the ad-hoc covariance matrix procedures will turn out to be ad-hoc useless. | |
| May 11, 2016 at 14:00 | comment | added | amoeba | @CagdasOzgenc: I don't think that was the issue raised by the OP. But I might be wrong. The question is now heavily edited, but you can take a look at the original formulation in the edit history. It was very unclear, at least to me. | |
| May 11, 2016 at 13:58 | comment | added | Cagdas Ozgenc | @amoeba I think the comments provided are not fulfilling. Yes you can calculate a covariance matrix for a data set as an arithmetic operation. The issue is I think whether doing this is meaningful in a non-multivariate normal distributional setting. | |
| May 11, 2016 at 12:56 | comment | added | amoeba | I edited your question (and upvoted). Please check that everything makes sense. | |
| May 11, 2016 at 12:55 | history | edited | amoeba | CC BY-SA 3.0 |
added 376 characters in body; edited title
|
| May 8, 2016 at 16:08 | comment | added | LeoW. | @amoeba: Reading the question as you formulated it, yes. Makes it probably way clearer :) | |
| May 7, 2016 at 21:19 | comment | added | amoeba | Great. However, your question remains very confusingly formulated. If my above comment is an answer to your question, then can your question be reformulated as "Given a data matrix X, can I always obtain its covariance matrix by subtracting the column means and computing XX'/n? Is this always valid?". Is this what you wanted to ask? | |
| May 6, 2016 at 22:16 | comment | added | amoeba | No restrictions, you can always subtract the mean to get the "centered" data and then $XX^\top/n$ will equal the covariance matrix. | |
| May 6, 2016 at 21:36 | comment | added | LeoW. | Okay, so maybe let me clear that question a bit: @amoeba, since you pointed out in one of your posts that the covariance matrix is equal to $XX^{\top}$ for centered data only. And we can center the data by subtracting the mean (correct?). Are there any restrictions when this can be applied? Since I should always be able to calculate the mean and subtract it. Since there exists a quantum algorithm which can calculate the PC exponentially faster (at least the theory) which works if the covariance matrix is given in the Gram Form. I wonder if there are any restrictions from that Form? | |
| May 6, 2016 at 20:17 | comment | added | amoeba | I still don't understand what you are asking, @LeoW. What exactly do you mean by "Gram matrix"? And what is your remaining question? If you are asking about the effect of centering on PCA, then it's a duplicate, see one of ttnphns's links above. | |
| May 6, 2016 at 19:56 | history | edited | LeoW. | CC BY-SA 3.0 |
removed first question due to comment
|
| May 6, 2016 at 19:55 | comment | added | LeoW. | @ttnphns, thanks for pointing out the question which addresses partly the first question. Here different issues are mentioned (slower SVD vs. numerical stability of the SVD). I see that it's actually implementation dependent as well. Since I am working on quantum algorithms i suppose this matter can hence be dismissed since it depends on the procedure used. Concerning question 2, I was referring to the case $R=XX^{\top}$ (as used in the first link). I updated the question and removed question 1. Thanks! | |
| May 6, 2016 at 16:07 | comment | added | ttnphns | @broncoAbierto, Let us not dispute about "standards" or ghosts. Btw, interestingly, Joliffe in his book discusses PCA on $X'X$, I remember to have seen it there. | |
| May 6, 2016 at 15:31 | comment | added | cangrejo | @ttnphns It is definitely interesting to consider the eigendecomposition of any matrix of the form $A^TA$. Is it standard, though? If I recall correctly, in Jolliffe, Ian. Principal component analysis. John Wiley & Sons, Ltd, 2002. PCs are specifically defined for the covariance matrix, as their purpose is to maximize the variance of the matrix columns when projected onto the span of the resulting subspaces. | |
| May 6, 2016 at 15:19 | comment | added | ttnphns |
@broncoAbierto, The principal components are defined as the eigenvectors of the covariance matrix. Not necessarily covariance. Linear PCs can be defined based on any SSCP-type matrix.
|
|
| May 6, 2016 at 10:46 | comment | added | cangrejo | @ttnphns The principal components are defined as the eigenvectors of the covariance matrix. Subtracting the expected value is not optional when speaking of covariance. | |
| May 6, 2016 at 8:44 | comment | added | ttnphns | Forth, in a broad sense, PCA - does not require centering; centering is seen as a pre-processing step. But results and their interpretation depend whether you center or not, of course. | |
| May 6, 2016 at 8:43 | comment | added | ttnphns | First of all, how do you define "Gram matix"? This very ambiguous, misused term sometimes mean here or there $X'X$ and sometimes $XX'$ and sometimes Gramian matrix in the sense "positive semidefinite" or some other sense... Second, in your first link I could not find any mentioning "Gram". Third, this question addresses the issue of precision in the dilemma "svd or eigen". | |
| May 6, 2016 at 8:19 | history | edited | LeoW. | CC BY-SA 3.0 |
deleted 18 characters in body
|
| May 6, 2016 at 8:14 | history | asked | LeoW. | CC BY-SA 3.0 |