Can PCA be applied for time series data?

Question

I understand that Principal Component Analysis (PCA) can be applied basically for cross sectional data. Can PCA be used for time series data effectively by specifying year as time series variable and running PCA normally? I have found that dynamic PCA works for panel data and the coding in Stata is designed for panel data and not time series. Is there any specific type of PCA which works on time series data?

Update. Let me explain in detail.

I am presently constructing an index for Infrastructure in India with variables like road length, rail route length, electricity generation capacity, number of telephone subscribers etc. I have 12 variables through 22 years for 1 country. Though I have reviewed papers that apply PCA on time series and even panel data, PCA is designed for cross sectional data which assumes i.i.d assumption. Panel and cross sectional data violates it and PCA does not take into account the time series dimension in it. I have seen dynamic PCA being applied only on panel data. I want to know if there is a specific PCA that is applied on time series or running static PCA with year defined as time series variable will do the job?

Answer 1

One approach could be to take the first time differences of your 12 variables to ensure stationarity. Then calculate the $12\times12$ covariance matrix and perform PCA on it. This will be some sort of average PCA over the whole time span, and will not say anything about how the different timelags affect each other. But it could be a good starting point.

If you are interested in decomposing the time domain as well I would check out SSA as suggested in the comments.

When you series are (assumed) stationary a single covariance matrix is meaningful. If your data is integrated of an order of 1 or higher, as I suspect they might be, the estimation of a single covariance matrix will not yield consistent results. A random walk is for example integrated of order 1, and the estimated covariance of two random walks does not say anything about their co-movement, here co-integration analysis is required.

As suggested in the comments PCA in itself doesn't care about stationarity so you can feed PCA any positive semi-definite matrix and the PC decomposition will be fine in a PCA-sense.

But if your estimated covariance matrix does not represent anything meaningful about the data, then PCA will, of course, not either.

Answer 2

Yes, PCA on time series is performed all the time in financial engineering (quantitative finance) and neurology.

In financial engineering, the data matrix is constructed with assets (e.g., stocks) in columns which represent the features, and the rows representing e.g. days (or objects) for end-of-day trading. Thus, the the data matrix $\underset{t \times p}{\bf X}$ has $t$ rows and $p$ columns. However, note that log-returns, $r_t=\log(P_t) - \log(P_{t-1}) = \log(P_t/P_{t-1})$, are used since daily prices are log-normally distributed -- i.e., skewed with right tails. Since there are 250 trading days/year, it's appropriate to fetch 1000 days of data which represents 4 years of trading. Since the same unit (e.g. USD) is usually used for daily log-price returns, the $p \times p$ covariance matrix for features is used for eigendecomposition. Otherwise, if different currencies are used, the correlation matrix is used for eigendecomposition, since correlation mean-zero standardizes the columns of $\bf{X}$. When done running PCA on assets, you can look at which stocks load on which PCs, a sort of clustering approach, or use the PC scores for input into other analyses. PCA is also run on the $t \times t$ covariance matrix for days, with assets in rows, in order to collapse days that correlate together into a single PC, since the general idea is that days can be redundant -- and when feeding data into e.g. a neural network, you don't want data rows to be redundant or features to be correlated (you want them to be orthogonal), since a neural net will waste time on learning the correlation. This approach does not focus on autocorrelation, however.

In quantitative finance, there is also a large interest in first finding the noise cutoff in eigenvalues of the covariance (correlation) matrix for many assets in order to improve (Markowitzian) portfolio optimization, since you want a portfolio that sits on the "efficient frontier" with assets that are uncorrelated. This approach exploits the Marcenko-Pastur law and the ratio $\gamma=t/n$ of the data matrix $\bf{X}$ for fitting the eigenvalue density, and finding the noise cuttoff known as $\lambda^+$, above which eigenvalues represent the signal, and below which eigenvalues represent noise. Once the noise eigenvalues are identified, the new dataset is based on (multivariate) regression of the original data on the PC scores representing the noise eigenvectors, $\mathbf{Y}=\mathbf{F}_n \beta$, and the residuals are then used as the denoised dataset, i.e., $\hat{\bf{X}}=\bf{Y}-\hat{\bf{Y}}$. Wealth values (cumulative return) from portfolios constructed using weights derived from the new dataset (residuals) have been shown to be much greater than without using this approach. Last, there's also a basic method to remove the "market effect" or widespread correlation among stock returns by regressing the asset data on the first PC representing the major (greatest) eigenvalue, $\mathbf{Y}=\mathbf{f}_1 \beta$, and pulling back the residuals to represent the new data, which will have the widespread market correlation removed. (since the first PC always represents stocks with high multicollinearity). This approach addresses market sentiment hinged to "herd-mentality."

In neurology, PCA is run on time-series for action potentials in different wavelength bands obtained from an EEG. Transforming the action potentials into orthogonal (uncorrelated) PC score vectors and inputting the PCs into other analyses is the primary means by which statistical power was increased in statistical genetic modelling of complex traits for behavioral genetics (since phenotypes for e.g. bi-polar, novelty-seeking, schizotypal, schozephrenia often overlap). The large Australian genetic twin studies were instrumental in parsing out these overlapping traits in behavioral genetics, because if there are disease differences among identical twins which are reared together (grow up in the same household), causal inference may point to exposure in different environments when they were older instead of their identical genetics. (identical twins "share 100% of their genes all the time").