0
$\begingroup$

I'm using R to get the principal components for several datasets.

An example result, using prcomp yields:

       PC1        PC2
X1 -0.7071068 -0.7071068
X2 -0.7071068  0.7071068

I'm then using the first principal component, from each dataset, as an input into another model (well, I plan to).

The problem is that, for interpretability, the direction of the first principal component matters. This is because I'll be multiplying my data by the components of the first eigenvector, and the direction changes the final result. (This result I also need to report).

If I get updated data, and retrain PCA, it's possible that the prcomp result could be:

       PC1        PC2
X1 0.7071068 -0.7071068
X2 0.7071068  0.7071068

And if I multiply by the first eigenvector, the result will be the additive inverse. I could just take the absolute value of the components of the first eigenvector, but this feels wrong... and also, I'm not sure about extending it to cases with more than 2 components.

(I know that if the first principal component was (0.7071068, 0.7071068), mathematically, the result would be the same; most variance explained along that line since eigenvectors multiplied by scalars are still eigenvectors.)

I'm curious if anyone has come across a similar problem, or if there is a technique for dealing with such issues?

For now, I am manually changing the direction of the first eigenvector on a case by case basis, if needed.

Any suggestions appreciated :)

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ It's not quite clear what the whole process is here. You have a bunch of datasets, you take PC1 of each, and input those to some other model. OK. What do you mean "If I get new data and rerun PCA"? Why would you get new data? Is it test data to which the model will be applied? Or is it new training data that you will use to retrain the model? In the former case you should not rerun PCA. In the latter case why do you care about signs matching to what you had before? $\endgroup$
    – amoeba
    Commented Jan 11, 2019 at 12:24
  • 2
    $\begingroup$ Direction (sign) is arbitrary in PCA. For convenience, often program assign sign so that in each column (component) of the loading matrix the sum is positive. I'm speaking about loadings, while you call eigenvector entries "loadings", please mind. $\endgroup$
    – ttnphns
    Commented Jan 11, 2019 at 13:24
  • 1
    $\begingroup$ @amoeba I'm looking at time series data. Each month, there will be a new set of observations. Each month, I want to retrain PCA, take the first eigenvector and reduce the dimensionality of the data down to one dimension. Since the direction of that first eigenvector is arbitrary, I could potentially see spurious large changes. $\endgroup$
    – orrymr
    Commented Jan 11, 2019 at 13:53
  • 1
    $\begingroup$ You can't take absolute values, that can change the span of the vector. You can multiply by a constant though (like -1). You could choose a normalization that always keeps the same direction (e.g. magnitude of 1 and positive first element, or first element equal to +1) but since you're claiming that the direction matters in your problem, it implies there is one correct normalization that is right for your problem. $\endgroup$
    – Dan
    Commented Jan 11, 2019 at 14:13
  • 1
    $\begingroup$ I see! You should have mentioned time series. Then it's almost a duplicate of stats.stackexchange.com/questions/34396. $\endgroup$
    – amoeba
    Commented Jan 11, 2019 at 14:19

0

Reset to default