3
$\begingroup$

I have time series data with five variables that have common variation and trends and they are very noisy. I want to extract their common variation (most likely the first principal component) and use it in a regression model.

Below is the original data: enter image description here

As you can see there is a strong correlation among the series, but they are very noisy.

Next I did PCA in R and extracted five components given below: enter image description here

I am a bit puzzled, should not the PCs behave like the original series in terms of trends i.e. sloping downward? Well, at least one that explains most variation?

Just in case, the R code I used is 

pcs=princomp(X[,2:6],cor=F)$scores
Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Well, one is trending -- the black one on your second plot, I'd bet it's the first PC. Maybe you are confused by the fact that it's trending upwards and not downwards? Well, the signs of the PCs are arbitrary, see stats.stackexchange.com/questions/88880. $\endgroup$
    – amoeba
    May 17, 2016 at 19:59
  • 1
    $\begingroup$ Yes, that's the case. I normalised the PC1 on its first observation and got the trend sloping downards. This was enough to convince me that there was nothing wrong. $\endgroup$
    – mr.rox
    May 18, 2016 at 14:15

2 Answers 2

Reset to default
2
$\begingroup$

If all original time series are trending downwards, then yes, you can expect the first PC to trend downwards as well. However, the signs of the PCs are arbitrary and do not have any meaning, see

The PC plotted in black on your second plot is trending upwards. I am sure it is the first PC and you are only confused because its sign is "wrong". You can flip it if you like, to make the first PC trend downwards too.

A sometimes useful convention is to fix the sign of the first component so that it is positively correlated to most variables. In your case this convention would result in the PC1 sloping downwards, as you expected.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ A sometimes useful convention... A widespread convenience is to assign sign to a component/factor loadings so that the sum of its loadings (with the sign, and not squared) be positive. $\endgroup$
    – ttnphns
    May 18, 2016 at 17:00
  • $\begingroup$ @ttnphns Thanks, this might be useful for me. What I am sometimes using is to select the sign of each factor such that there is more positive loadings than negative ones. But your convention might make a bit more sense, as it takes into account the magnitude of each loading value. I will try it. $\endgroup$
    – amoeba
    May 18, 2016 at 19:13
1
$\begingroup$

There are several choices which must be made before running PCA on time series. First, PCA should be run using the covariance matrix and not the correlation matrix, according to what most reports suggest when running PCA on time series. The choice of covariance vs. correlation will typically introduce slight amplitude as well as phase shift differences in the resulting PC scores.

Second, the standardized (scaled) PC scores should be used and not the non-standardized (unscaled) PC scores. Standardized scores are based on the loadings, which will scale the scores to the inverse of the eigenvalues, i.e., $1/\sqrt{\lambda_j}$. The raw scores can have much larger range and scale, and for the jumpy data you have, results could yield large transitions, whereas the standardized will be tighter with much lower range.

Overall, it is not recommended that you simply generate PC scores without specifying the choices described above.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.