23
$\begingroup$

I understand that Principal Component Analysis (PCA) can be applied basically for cross sectional data. Can PCA be used for time series data effectively by specifying year as time series variable and running PCA normally? I have found that dynamic PCA works for panel data and the coding in Stata is designed for panel data and not time series. Is there any specific type of PCA which works on time series data?

Update. Let me explain in detail.

I am presently constructing an index for Infrastructure in India with variables like road length, rail route length, electricity generation capacity, number of telephone subscribers etc. I have 12 variables through 22 years for 1 country. Though I have reviewed papers that apply PCA on time series and even panel data, PCA is designed for cross sectional data which assumes i.i.d assumption. Panel and cross sectional data violates it and PCA does not take into account the time series dimension in it. I have seen dynamic PCA being applied only on panel data. I want to know if there is a specific PCA that is applied on time series or running static PCA with year defined as time series variable will do the job?

$\endgroup$
  • 2
    $\begingroup$ Singular Spectrum Analysis (SSA) is often called PCA for time series. en.wikipedia.org/wiki/Singular_spectrum_analysis $\endgroup$ – Vladislavs Dovgalecs Jun 23 '15 at 15:30
  • 1
    $\begingroup$ Please review some of the posts in the sidebar to the right (-->) that refer to both PCA and time series. If any answer your question, please link to it in comments here, but if none do, you might explain the specific ways in which your issue differs from any of those. $\endgroup$ – Glen_b Jun 24 '15 at 5:27
  • $\begingroup$ None of them answers the question of pca on time series. Specific queries on the topic either relates to science or are left unanswered. $\endgroup$ – Nisha Simon Jun 24 '15 at 7:13
  • 5
    $\begingroup$ PCA, as a data transformation, dimensionality reduction, exploration, and visualization tool, does not make any assumptions. You can run it on any data whatsoever, including time series data. In fact, PCA is very often applied for time series data (sometimes it is called "functional PCA", sometimes not). I don't even know what "dynamic PCA" and "static PCA" should mean; don't worry and use standard PCA. $\endgroup$ – amoeba Jun 28 '15 at 13:11
  • $\begingroup$ You may wanna consider using Functional PCA which is particularly designed for time series. The FDA package in R implemented fPCA. You will be able to find the multivariate fPCA. $\endgroup$ – Anne Jul 7 '18 at 23:47
9
$\begingroup$

One approach could be to take the first time differences of your 12 variables to ensure stationarity. Then calculate the $12\times12$ covariance matrix and perform PCA on it. This will be some sort of average PCA over the whole time span, and will not say anything about how the different timelags affect each other. But it could be a good starting point.

If you are interested in decomposing the time domain as well I would check out SSA as suggested in the comments.

When you series are (assumed) stationary a single covariance matrix is meaningful. If your data is integrated of an order of 1 or higher, as I suspect they might be, the estimation of a single covariance matrix will not yield consistent results. A random walk is for example integrated of order 1, and the estimated covariance of two random walks does not say anything about their co-movement, here co-integration analysis is required.

As suggested in the comments PCA in itself doesn't care about stationarity so you can feed PCA any positive semi-definite matrix and the PC decomposition will be fine in a PCA-sense.

But if your estimated covariance matrix does not represent anything meaningful about the data, then PCA will, of course, not either.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ +1. What do you mean by "the first time differences"? $\endgroup$ – amoeba Jun 28 '15 at 14:14
  • $\begingroup$ I mean the first difference, so for each of the twelve x's I would do x_t - x_t-1. $\endgroup$ – Duffau Jun 28 '15 at 14:22
  • $\begingroup$ So you suggest to do PCA on time derivatives of each time series, as opposed to time series themselves. That's interesting; why would this be your first suggestion? $\endgroup$ – amoeba Jun 28 '15 at 14:45
  • $\begingroup$ For two reasons: 1) For covariance estimation to be consistent, the normal cross sectional assumptions is for the two random variables to be independent and identically distributed (iid). This ensures the convergence of the sample mean to expected value, the so called Law of large numbers (LLN). In time series analysis the assumption of two stochastic processes being iid is to restrictive. So it is replaced with the notion of stationarity (of many different kinds). For the LLN to hold and covariance estimation to be consistent the two series need to have a jointly stationary distribution. $\endgroup$ – Duffau Jun 28 '15 at 21:54
  • $\begingroup$ If each stochastic process is staionary then (I'm allmost positive that) they have are jointly stationary, hence covariance estimation makes sense. First differences is a standard technique in econometrics to make time series "more stationary". And from here estimation and PCA is straight forward. So in short, because it is easy :-) .... ok there was no second reason.. $\endgroup$ – Duffau Jun 28 '15 at 22:00
2
$\begingroup$

Yes, PCA on time series is performed all the time in financial engineering (quantitative finance) and neurology.

In financial engineering, the data matrix is constructed with assets (e.g., stocks) in columns which represent the features, and the rows representing e.g. days (or objects) for end-of-day trading. Thus, the the data matrix $\underset{t \times p}{\bf X}$ has $t$ rows and $p$ columns. However, note that log-returns, $r_t=\log(P_t) - \log(P_{t-1}) = \log(P_t/P_{t-1})$, are used since daily prices are log-normally distributed -- i.e., skewed with right tails. Since there are 250 trading days/year, it's appropriate to fetch 1000 days of data which represents 4 years of trading. Since the same unit (e.g. USD) is usually used for daily log-price returns, the $p \times p$ covariance matrix for features is used for eigendecomposition. Otherwise, if different currencies are used, the correlation matrix is used for eigendecomposition, since correlation mean-zero standardizes the columns of $\bf{X}$. When done running PCA on assets, you can look at which stocks load on which PCs, a sort of clustering approach, or use the PC scores for input into other analyses. PCA is also run on the $t \times t$ covariance matrix for days, with assets in rows, in order to collapse days that correlate together into a single PC, since the general idea is that days can be redundant -- and when feeding data into e.g. a neural network, you don't want data rows to be redundant or features to be correlated (you want them to be orthogonal), since a neural net will waste time on learning the correlation. This approach does not focus on autocorrelation, however.

In quantitative finance, there is also a large interest in first finding the noise cutoff in eigenvalues of the covariance (correlation) matrix for many assets in order to improve (Markowitzian) portfolio optimization, since you want a portfolio that sits on the "efficient frontier" with assets that are uncorrelated. This approach exploits the Marcenko-Pastur law and the ratio $\gamma=t/n$ of the data matrix $\bf{X}$ for fitting the eigenvalue density, and finding the noise cuttoff known as $\lambda^+$, above which eigenvalues represent the signal, and below which eigenvalues represent noise. Once the noise eigenvalues are identified, the new dataset is based on (multivariate) regression of the original data on the PC scores representing the noise eigenvectors, $\mathbf{Y}=\mathbf{F}_n \beta$, and the residuals are then used as the denoised dataset, i.e., $\hat{\bf{X}}=\bf{Y}-\hat{\bf{Y}}$. Wealth values (cumulative return) from portfolios constructed using weights derived from the new dataset (residuals) have been shown to be much greater than without using this approach. Last, there's also a basic method to remove the "market effect" or widespread correlation among stock returns by regressing the asset data on the first PC representing the major (greatest) eigenvalue, $\mathbf{Y}=\mathbf{f}_1 \beta$, and pulling back the residuals to represent the new data, which will have the widespread market correlation removed. (since the first PC always represents stocks with high multicollinearity). This approach addresses market sentiment hinged to "herd-mentality."

In neurology, PCA is run on time-series for action potentials in different wavelength bands obtained from an EEG. Transforming the action potentials into orthogonal (uncorrelated) PC score vectors and inputting the PCs into other analyses is the primary means by which statistical power was increased in statistical genetic modelling of complex traits for behavioral genetics (since phenotypes for e.g. bi-polar, novelty-seeking, schizotypal, schozephrenia often overlap). The large Australian genetic twin studies were instrumental in parsing out these overlapping traits in behavioral genetics, because if there are disease differences among identical twins which are reared together (grow up in the same household), causal inference may point to exposure in different environments when they were older instead of their identical genetics. (identical twins "share 100% of their genes all the time").

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.