# Applying PCA on time series when some time series have to be differenced and some not

I am currently trying to perform PCA on time series data and I'm having some confusion. I need to make my data stationary first before I can perform PCA on it and here is the confusion.

I have 128 observations for each time series but some time series require second differencing while some others require just a single one. So after differencing, the resulted time series will be of different length (126 and 127).

Then how do I get $n \times t$ matrix since some observations are missing from the differencing?

• The meaningfulness of applying PCA to totally different kinds of objects, such as first differences of values and second differences of values, is doubtful. Could you explain what you're hoping to accomplish with PCA? What problem(s) do you hope it will solve?
– whuber
Aug 23, 2016 at 17:47
• Hi @whuber, I am actually trying to use PCA to extract the factors for forecasting using factor model. im trying to do something like stock and watson (2002) Forecasting Using Principal Components From a Large Number of Predictors Aug 23, 2016 at 17:53
• Can you please capitalize first letters of each sentence, make pronoun "I" capital, put apostrophes in "I'm", and in general adhere to English punctuation? This will make your question much more pleasant to read. Even if you are not a native English speaker, your English is definitely good enough so that such trivialities should not be a problem to you. Thanks. Aug 23, 2016 at 18:28
• @amoeba corrected! Aug 23, 2016 at 18:30
• When you say some time series "require" differencing, this seems to imply your model requires stationary time series. Can you specify what time series model you want to estimate? Aug 23, 2016 at 18:54

Since you are doing dimensionality reduction already (via PCA), then a simple thing to do would be to use the shorter length as your base, and split any longer series into multiple (redundant) series.

Say you start with two time series, $x_t$ and $y_t$, where $t=1,2,\ldots,n$. However $y$ has a secular trend, so you want to difference it, and use $\Delta y_t=y_{t+1}-y_t$ instead. A simple option is to increase the dimensionality of your "data" from 2 to 3, adding a "look ahead" version of $x$. So now your data would be $x_t$, $x_{t+1}$, and $\Delta y_t$, where now $t=1,2,\ldots,n-1$.

Here the idea would be that if the two $x$ series have the "same information", then the PCA will tell you this.

Without knowing more about your particular data and application, I cannot say whether or not this technique would be appropriate (e.g. in terms of causality or your system dynamics).

• An interesting idea. Can you cite references where it is used. Aug 25, 2016 at 22:58
• @horaceT I cannot cite any references. Broadly, I kind of had the idea of delay embedding in mind. This is a nonlinear dynamics technique to "reconstruct topology of strange attractors" using lagged versions of one measured variable to "fill in" for unobserved variables. Aug 28, 2016 at 6:32

From a purely technical perspective, the common remedy for unequal time series lengths used in applications is

1. to match time indices of the different series in a sensible way (e.g. match $x_t$ with $y_t$, not with $y_{t-h}$ for $h\neq 0$, unless the subject-matter logic suggests otherwise) and then
2. to cut the values that stick out in the beginning or the end of the series.

This way you are throwing out data, but often it is a minor loss (in your case, you would only lose 1 observation). For example, this is typically done in AR and VAR models where lags of the original variables are shorter series than the original series. (Note that in AR and VAR models the time indices across the original and the lagged series are matched with time lags intentionally and the data matrix looks like $\{(y_t, y_{t-1},\dotsc,y_{t-p})\}_{t=p+1}^T$, unlike what I mention in point 1.)

But there still remains a question how to interpret PCA for variables transformed using different orders of diferencing...