Time series forecast by Principal Component Analysis

Question

Suppose that I have a series of $M$ time-observations of $N$ "quantities" $z_1(t_1),...,z_1(t_M)$, ..., $z_N(t_1),...,z_N(t_M)$. I want to estimate the values of $z_1(t_{M+1}),...,z_N(t_{M+1})$. This a problem of interest, for example, in stock asset prediction. I want to use Principal Component Analysis (PCA) performed by a Singular Value Decomposition (SVD).

My questions:

1. What is the physical meaning of the first singular vector (namely, that one corresponding to the largest singular value)? What is the physical meaning of the remaining singular vectors? I understand that the singular vectors provide uncorrelated linear combinations of the above random variables.
2. What is the physical meaning of the singular values? As long as I know, they are related to the variances associated to the singular vectors. But then, does it mean that the most probable value of the quantities at time $t_{M+1}$ is due to the least singular values (least variances)?
3. How do I use the singular values and vectors of PCA to predict the value of the quantitites at time $t_{M+1}$? Could you let me understand the idea behind?

Thank you very much in advance.

Answer 1

I think your intuition is in the right direction, but what you're missing is that it is customary to subtract the mean smaple vector before performing PCA. The PCA vectors then represent deviations from the mean. The importance of the components in PCA analysis is always from the one with highest value/score to the one with lowest value/score. Depending on the particular application, the vectors with lowest associated values may even be omitted. The intuition is that to understand the variations in the data between samples it is often easier to think in terms of "independently occurring" (more precisely, uncorrelated) variations rather than variations of each specific element, if the elements actually vary together (and in many problems areas, financial assets included, they tend to vary together).

Answer 2

Principal component analysis performs high-dimensional rotation of the source data to maximize its variance. The singular vectors represent the basis of the rotated coordinate system and the principal components are the source data projected onto this basis. The singular values represent the variances of the principal components.

I'd say that PCA-based prediction should look like extrapolating the principal components and projecting them back to the original coordinate system. I don't know PCA-based prediction methods exactly, but I can point you to two related (via SVD) prediction methods:

1. Kumaresan-Tufts linear prediction (ref.)
2. SSA forecasting (N. Golyandina et al.)

Answer 3

You mention stock returns. In that case the first few eigenvalues/singular values of the PCA turn out to have an explicit interpretation:

1. The first singular value is closely related to the market portfolio: the eigenvector is a linear combination of the $z_i(t)$ with positive coefficients
2. The next couple (say 5) of eigenvalues are typically related to style or sector portfolios, such as defensive vs. cyclical stocks. These stocks often go in and out of favor as a group, which shows up in a principal component decomposition.

Beyond that there is noise, so the last singular value (your question #2) will not tell you anything.

As for your question #3, you can for example try out a predictive regression, as discussed in this paper by Stambaugh