2
$\begingroup$

I'm trying to implement Principal Component Analysis (PCA) in a portfolio replication procedure. (The replication procedure looks like regression: there is a vector representing payoffs of an asset under different ecocnomic scenarios, I need to find a linear combination of vectors of another assets that fits the most closely to an initial asset).

Coefficients I get from regression should tell me how many assets I need to buy and sell to get approximately the same payoff as the asset I'm trying to replicate.

There is a problem with regression as the candidate asset matrix is ill-conditioned: assets show high correlation. There's a hope that orthogonal principal components could resolve this problem.

I need to do a PCA decomposition of candidate assets matrix, take only first $n$ components, do optimization and get components coefficients. Then I need to transform the coefficients back into an original basis.

Now the problem: PCA usually works with mean-centered data, but if I subtract means from the original data, I don't know how to interpret resulting coefficients in my case and don't know how to reverse the operation.

So far I'm doing eigen-decomposition of a covariance matrix, then using eigenvectors to make an orthogonal transformation of the data that is not mean-centered. Then I'm running a regression (actually L1 norm optimization) to get coefficients and transform them back into an original basis. The results are not bad, but I can't stop thinking about the problem with mean-centering, if I'm doing it completely wrong.

I was hoping to find a detailed math reasoning for this problem, but unfortunately failed. I'm very much a noob in this and my math skills are far from being good, so I would really appreciate your help if you can share some insights on the problem of mean-centering in the PCA.

Improve this question
$\endgroup$
4
  • $\begingroup$ In the current state this is sooner a discourse with highly uncommon terminology than a question. I'm, particularly, in difficulty to make head or tail of it. Can you make the story clearer? $\endgroup$
    – ttnphns
    Oct 21, 2013 at 0:24
  • $\begingroup$ @ttnphns Thanks for pointing it out. I was really using too much of terminology. Hope now it's better. I tried to make the problem description clearer. Though I understand that it's still messy. I'm working on it. $\endgroup$
    – Anton
    Oct 21, 2013 at 12:32
  • $\begingroup$ I'm doing eigen-decomposition of a covariance matrix, then using eigenvectors to make an orthogonal transformation of the data that is not mean-centered Are you saying that you compute the PC scores by multiplying raw_dataeigenvectors, not centered_dataeigenvectors? (That gives the PCs which are completely correlated with the "true", centered PCs.) $\endgroup$
    – ttnphns
    Oct 21, 2013 at 13:29
  • $\begingroup$ @ttnphns Exactly. I plotted toy data in 2d and checked the rotation. It works the same as if data was mean-centered, but just a pivot point of rotation is not located in the mean (honestly, I don't know where it is). My concern now is if it can affect optimization results in some way. After receiving optimized coefficients, i just multiply them by transposed eigenvectors. $\endgroup$
    – Anton
    Oct 21, 2013 at 13:50

2 Answers 2

Reset to default
2
$\begingroup$

Unstandardized principal component values (PC scores) are given by $\bf XU$, where $\bf X$ is the data matrix and $\bf U$ is the matrix of eigenvectors. If covariance matrix decomposed to $\bf U$, - then $\bf X$ must be centered to give proper PC scores. But you said that you used raw data to compute the PC scores despite that eigenvectors came from covariance matrix. What happens then?

Example is below. Variables are V1, V2. PC1 and PC2 are the scores computed in the usual way; these are centered PC scores. dcPC1 and dcPC2 are the scores computed by "your" way; these are decentered PC scores.

      V1       V2      PC1      PC2    dcPC1    dcPC2

  6.7662   8.6155   2.8843    .3930  10.9224    .8427
  5.9534   6.9533   1.0506    .6401   9.0887   1.0897
  5.1772   4.6352  -1.3083   1.2819   6.7298   1.7315
  5.3906   3.5785  -2.0685   2.0461   5.9696   2.4958
  3.0136   6.5524   -.9154  -1.5821   7.1227  -1.1325
  1.4195   3.8332  -4.0620  -1.3978   3.9761   -.9481
  4.9248   6.0971   -.2327    .2602   7.8054    .7098
  4.5031   8.5152   1.5441  -1.4333   9.5822   -.9837
  6.0504   6.8867   1.0491    .7578   9.0872   1.2074
  1.7513   2.2287  -5.2121   -.2308   2.8260    .2188
  4.4432   7.1862    .4056   -.7451   8.4437   -.2955
  2.8280   5.4160  -1.9635  -1.1054   6.0746   -.6558
  6.8661   3.4229  -1.3786   3.3597   6.6595   3.8093
  3.6724   3.9823  -2.6869    .3930   5.3512    .8426
  5.8395   6.4047    .5311    .8500   8.5692   1.2996
  6.7118  11.4956   5.2492  -1.2516  13.2873   -.8020
  4.7179   8.8247   1.9208  -1.4266   9.9589   -.9770
  1.0230   3.2331  -4.7813  -1.3943   3.2568   -.9446
  7.2815  10.1138   4.4165   -.0105  12.4546    .4391
  8.4197  10.7265   5.5581    .5958  13.5962   1.0454

The pic displays the principal components drawn in the space of the variables. In both computational cases, the PCs are the same - as axes. They orthogonally cross each other in the centre of the cloud. This is because covariance matrix was analyzed (it implies centering). However, the scores (shown as markers on the PC axes) are different. "Decentered PC1" = "Centered PC1" + 8.04. "Decentered PC2" = "Centered PC2" + 0.45. It is unclear what use could be of such decentered scores, since they do not share their means with each other and with the data cloud.

enter image description here

Improve this answer
$\endgroup$
0
0
$\begingroup$

@ttnphns Thank you for your answer. You're right, in common case, without mean-centering PCA would lose a lot of sense.

But I think, if I use it to regularize regression, it could work.

Here I wrote a bit of algebra for myself. I hope it's correct, but I would appreciate some comments, cause I'm not completely sure.

  1. Initial problem (doesn't work because A'A is nearly collinear and numerically uninvertible) $$ y = Ax $$

  2. Decompose covariance matrix of data (standardized data). To get correct (from the PCA perspective) eigen-everything. $$ cov(A) = VDV' $$

  3. Transform raw-data (without mean centering) into an orthogonal basis $$ AV $$ (just a linear transformation).

  4. Coefficients $x^*$ for transformed problem are founded because matrix $AV$ is well-conditioned. $$ y = AVx^* $$
  5. As (1) and (4) are approximately equal $$ Ax = AVx^* $$
  6. so... $$ x = (A'A)^{-1}A'AVx^* $$
  7. A'A seems to have a full rank, so theoretically is invertible analiticaly $$ x = Vx^* $$
Improve this answer
$\endgroup$
2
  • $\begingroup$ From your (unusual) notatition one may see that A is the multivariate data matrix and x are the vector of regression coefficients. A implies a rectangular matrix. How can a rectangular matrix ever be invertible or uninvertible (unless we speak of generalized inversion)? In (6) you invert it, uninvertible by your own words. (7) can't follow from (6) because A^-1 * A would have been I, identity matrix, and not 1. $\endgroup$
    – ttnphns
    Oct 22, 2013 at 8:14
  • $\begingroup$ @ttnphns Oh, you right, that was stupid. I wanted to say that A'A is practically non-invertible because of numerical errors. But theoretically it still has a full rank. Can't I premultiply 5 by A' to get A'Ax = A'AVx* and then premultiply A'A by its inverse to cansel them? I'm doing inverse analiticaly, not numerically, so should I consider it to be a problem? p.s.: Sorry for notation. It's my first question here. Probably, I need some time to figure out how it works. $\endgroup$
    – Anton
    Oct 22, 2013 at 10:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.