What does "performing PCA on a single time series" mean/do?

Question

Yes, I already looked here but that's too high profile for my humble mind (and it's not exactly what I'm looking for).

Imagine we have a timecourse with time on the x-axis and some value on the y-axis (e.g. a signal). Now I can sample this timecourse to obtain a vector in a multidimensional vector space.

My question: What does it mean if I perform a PCA on this data? What is the PCA of a single vector (such as the timeseries) and how can I interpret the resulting eigenvectors?

Answer 1

It isn't meaningful to run PCA on a univariate time series (or, more generally, a single vector). To run PCA on time series data, you'd need to have either a multivariate time series, or multiple univariate time series. There are ways to transform a univariate time series into a multivariate one (e.g. wavelet or time-frequency transforms, time delay embeddings, etc.). For example, the spectrogram of a univariate time series gives you the power at each frequency, for each moment in time.

Say we have a multivariate time series with $p$ dimensions/variables. Or, we might have a set of $p$ univariate time series, where each time point has some common meaning across time series (e.g. time relative to some event). In both cases, there are $n$ time points. There are a couple ways to run PCA:

1. Consider each time point to be an observation. Dimensions correspond to variables of the multivariate time series, or to the different univariate time series. So, there are $n$ points in a $p$ dimensional space. In this case, eigenvectors correspond to instantaneous patterns across the dimensions/time series. At each moment in time, we represent the amplitude across dimensions/time series as a linear combination of these patterns.

2. Consider each variable of the multivariate time series (or each univariate time series) to be an observation. Dimensions correspond to time points. So, there are $p$ points in an $n$-dimensional space. In this case, the eigenvectors correspond to temporal basis functions, and we're representing each time series as a linear combination of these basis functions.

Given the above, it's apparent why PCA doesn't make sense for a single univariate time series. Either you have $n$ observations and 1 dimension (in which case there's nothing for PCA to do), or you have a single observation with $n$ dimensions (in which case the problem is completely underdetermined and all solutions are equivalent).

Answer 2

PCA on a single time series can be done, of course. The result will be one principal component, which will be equal to the original series. Hence, technically it'll work, but it'll be pointless: you'll get your input series in the output.

Here's a MATLAB example. I got the PCA of a random series, then plotted the only principal component against the original series to show that it's the same thing. I also show you the differences between to series (adjusted for mean) is zero.

x=randn(10,1);
[~,score,~,~,~,mu]=pca(x);
scatter(x,score);
max(abs(x-score-mu))

ans =

   4.4409e-16

Answer 3

Maybe, “performing PCA on a single time series” means application of singular spectrum analysis (SSA), which is sometimes called PCA of time series. In SSA, multivariate data are constructed from lagged (moving) subseries of the initial time series. Then PCA (usually, SVD, which is PCA without centering/standardizing) is applied to the obtained multivariate data.

Example in R:

library(Rssa)
s <- ssa(co2)
# Plot eigenvectors
plot(s, type = "vectors")
# Reconstruct the series, grouping elementary series.
r <- reconstruct(s, groups = list(Trend = c(1, 4), Season1 = c(2,3), Season2 = c(5, 6)))
plot(r)

Answer 4

Let us assume that whoever was asking you to perform PCA on a univariate time series was really asking you "How many linearly independent subsegments does this single time series have?". We can perform PCA on this data after a simple transformation of your univariate time series. Assume your time series has length $T$ and that $T=MN$, where $M$ is the length of each temporal subsegment of the original time series. You can perform PCA on this $M$ x $N$ matrix. For a very contrived example, say you have this time series with time on x-axis and total time length of 750:

Let us now rearrange this vector of length $T$ to a 250 x 3 matrix. Notice in this very contrived case that the waveform repeats itself every 250 timesteps. We then run PCA on this 250 x 3 matrix. The plot of the variance explained vs the PCA components ranked by variance explained will be:

We can plot the representation of our original data in the PC space, where we see the M-length vector corresponding to PC1 (blue) matches with the actual observed waveform that repeats, and that the plot of the M-length vector corresponding to PC2 (gold) is essentially zero as all of the variance in the original time series is explained by PC1.

This is greatly simplified versus how you would do this in application. The best way to do it would be to have $M$ act as a sliding filter along the time series, where the length of $M$ and the stride are hyperparameters. You could use PCA on the eventual $MN$ matrix to reconstruct the original signal, treating the PCA scores as a basis set and comparing error between your reconstruction and the original signal. In this basic example, the stride is equal to the length of $M$