# How to interpret PCA on time-series data?

I am trying to understand the use of PCA in a recent journal article titled "Mapping brain activity at scale with cluster computing" Freeman et al., 2014 (free pdf available on the lab website). They use PCA on time series data, and use the PCA weights to create a map of the brain.

The data is trial-average imaging data, stored as a matrix (called $\hat {\mathbf Y}$ in the paper) with $n$ voxels (or imaging locations in the brain) $\times \hat t$ time points (the length of a single stimulation to the brain).

They use the SVD resulting in $$\hat {\mathbf Y} = \mathbf{USV}^\top$$ ($\mathbf V^\top$ indicating transpose of matrix $\mathbf V$).

The authors state that

The principal components (the columns of $\mathbf V$) are vectors of length $\hat t$, and the scores (the columns of $\mathbf U$) are vectors of length $n$ (number of voxels), describing the projection of each voxel on the direction given by the corresponding component, forming projections on the volume, i.e. whole-brain maps.

So the PCs are vectors of length $\hat t$. How can I interpret that the "first principal component explains the most variance" as is commonly expressed in tutorials of PCA? We started with a matrix of many highly correlated time-series -- how does a single PC time series explain variance in the original matrix? I understand the whole "rotation of a Gaussian cloud of points to the most-varied axis" thing, but am unsure how this relates to time-series. What do the authors mean by direction when they state: "the scores (the columns of $\mathbf U$) are vectors of length $n$ (number of voxels), describing the projection of each voxel on the direction given by the corresponding component"? How can a principal component time course have a direction?

To see an example of the resulting times series from linear combinations of principle components 1 and 2 and the associated brain map, go to the following link and mouse over on the the dots in the XY plot. My second question is related to the (state-space) trajectories they create using the principal component scores.

These are created by taking first 2 scores (in the case of the "optomotor" example I have outlined above) and project the individual trials (used to create the trial-averaged matrix described above) into the principal subspace by the equation: $$\mathbf J = \mathbf U^\top \mathbf Y.$$

As you can see by the linked movies, each trace in state space represents the activity of the brain as a whole.

Can someone provide the intuition for what each "frame" of the state space movie means, as compared to the figure that associates the XY plot of the scores of the first 2 PCs. What does it means at a given "frame" for 1 trial of the experiment to be in 1 position in the XY state-space and another trial to be in another position? How do the XY plot positions in the movies relate to the principle component traces in the linked figure mentioned in the first part of my question? • +1 I edited your question, take a look at how one can format tex equations here. Apart from that, I know the paper quite well, so will reply later. Nov 25, 2014 at 20:01
• This is not exactly what the OP wants, but might become handy in interpreting the principal components when taken from time-series data, as I do this all the time. I usually like to interpret PCA as a Karhunen-Loève expansion: expressing a given time series, $X_t$ (the different time-series that you apply PCA to), as a linear combination of uncorrelated time series (i.e., the principal components). The weights of each time series in this case are given by the eigenvectors obtained from the covariance matrix. Nov 25, 2014 at 21:02
• (See this for a more in-depth explaination of my point: astro.puc.cl/~nespino/files/Ch2_PCA_nespinoza.pdf) Nov 25, 2014 at 21:02
• I added to your question some screenshots that you were referring to. Nov 26, 2014 at 12:07
• Link from @statHacker didn't work for me. Here is one that I found working web.archive.org/web/20170517051143/http://www.dsea.unipi.it/… Aug 15, 2019 at 6:48

## Q1: What is the connection between PC time series and "maximum variance"?

The data that they are analyzing are $\hat t$ data points for each of the $n$ neurons, so one can think about that as $\hat t$ data points in the $n$-dimensional space $\mathbb R^n$. It is "a cloud of points", so performing PCA amounts to finding directions of maximal variance, as you are well aware. I prefer to call these directions (which are eigenvectors of the covariance matrix) "principal axes", and the projections of the data onto these directions "principal components".

When analyzing time series, the only addition to this picture is that the points are meaningfully ordered, or numbered (from $1$ to $\hat t$), as opposed to being simply an unordered collection of points. Which means that if we take firing rate of one single neuron (which is one coordinate in the $\mathbb R^n$), then its values can be plotted as a function of time. Similarly, if we take one PC (which is a projection from $\mathbb R^n$ on some line), then it also has $\hat t$ values and can be plotted as a function of time. So if original features are time series, then PCs are also time series.

I agree with @Nestor's interpretation above: each original feature can be then seen as a linear combination of PCs, and as PCs are uncorrelated between each other, one can think of them as basis functions that the original features are decomposed into. It's a little bit like Fourier analysis, but instead of taking fixed basis of sines and cosines, we are finding the "most appropriate" basis for this particular dataset, in a sense that first PC accounts for most variance, etc.

"Accounting for most variance" here means that if you only take one basis function (time series) and try to approximate all your features with it, then the first PC will do the best job. So the basic intuition here is that the first PC is a basis function time series that fits all the available time series the best, etc.

## Why is this passage in Freeman et al. so confusing?

Freeman et al. analyze the data matrix $\hat{\mathbf Y}$ with variables (i.e. neurons) in rows (!), not in columns. Note that they subtract row means, which makes sense as variables are usually centred prior to PCA. Then they perform SVD: $$\hat {\mathbf Y} = \mathbf{USV}^\top.$$ Using the terminology I advocate above, columns of $\mathbf U$ are principal axes (directions in $\mathbb R^n$) and columns of $\mathbf{SV}$ are principal components (time series of length $\hat t$).

The sentence that you quoted from Freeman et al. is quite confusing indeed:

The principal components (the columns of $\mathbf V$) are vectors of length $\hat t$, and the scores (the columns of $\mathbf U$) are vectors of length $n$ (number of voxels), describing the projection of each voxel on the direction given by the corresponding component, forming projections on the volume, i.e. whole-brain maps.

First, columns of $\mathbf V$ are not PCs, but PCs scaled to unit norm. Second, columns of $\mathbf U$ are NOT scores, because "scores" usually means PCs. Third, "direction given by the corresponding component" is a cryptic notion. I think that they flip the picture here and suggest to think about $n$ points in $\hat t$-dimensional space, so that now each neuron is a data point (and not a variable). Conceptually it sounds like a huge change, but mathematically it makes almost no difference, with the only change being that principal axes and [unit-norm] principal components change places. In this case, my PCs from above ($\hat t$-long time series) will become principal axes, i.e. directions, and $\mathbf U$ can be thought as normalized projections on these directions (normalized scores?).

I find this very confusing and so I suggest to ignore their choice of words, but only look at the formulas. From this point on I will keep using the terms as I like them, not how Freeman et al. use them.

## Q2: What are the state space trajectories?

They take single-trial data and project it onto the first two principal axes, i.e. the first two columns of $\mathbf U$). If you did it with the original data $\hat{\mathbf Y}$, you would get two first principal components back. Again, projection on one principal axis is one principal component, i.e. a $\hat t$-long time series.

If you do it with some single-trial data $\mathbf Y$, you again get two $\hat t$-long time series. In the movie, each single line corresponds to such projection: x-coordinate evolves according to PC1 and y-coordinate according to PC2. This is what is called "state space": PC1 plotted against PC2. Time goes by as the dot moves around.

Each line in the movie is obtained with a different single trial $\mathbf Y$.

• I asked this question as a comment to below, but perhaps @amoeba can help? Is the first principal component weights vector just the mean time series collapsing across all voxels? If it were the mean, it would result in the smallest scores to fit to the individual data traces. – Dec 2, 2014 at 13:54
• The short answer is no, it is generally not the mean time series, although in many cases it can be quite close. As an example, think of a collection of time series that are all straight lines with different slopes (positive and negative) all going through zero. Then the mean time series is around constant zero. But the first PC will be strong linear line. BTW, I think this is an excellent question and if you want more details and/or figures, please ask it (again) as a separate question. Just be sure not to duplicate any parts of this question about Freeman et al.; make them separate. Dec 2, 2014 at 16:22
• (or anyone else interested in a response) - with regards to Q2, what do you mean by "project [each trial] onto the first two [PCs]". Mathematically it is very clear that U is a vector of length n voxels, and when matrix multiplied by the length n matrix Y we achieve dimensionality reduction to the 1st 2 PCs. Can you provide intuition with regards to U being the matrix of scores (i.e. the distance of each voxel from the first 2 PCs). Can I think of each timepoint of J as the 2-d average of the projection of each voxels position in the 2 dimensional plot of the 1st image above? Dec 3, 2014 at 14:53
• You are right, I had this badly explained. I edited my question, expanding several parts of it; please read again from the beginning (or see what I changed) to make sure you follow. Don't think about $U$ as "scores"!! $U$ are principal axes, directions in the neuron space. They take single trials and project on the two first principal axes. What they get (from one trial) is two time series. They could plot both as a function of time, but instead they plot them one against another: PC1 vs PC2. This is "state space". Each line corresponds to different single trial! So to your last question: NO! Dec 3, 2014 at 22:22
• Is “$\mathbf{SV}$” a typo above?: "Using the terminology I advocate above, columns of U are principal axes (directions in Rn) and columns of SV are principal components (time series of length t^)." Dec 4, 2014 at 17:04

With respect to the first question. Consider the whole time series through a particular voxel to be a single draw from a multivariate distribution. We can now think of this as a multivariate vector much like any other that we might apply PCA to. The first $p$ columns of $\bf V$ are then the eigen-timecourses which, when linearly combined provide the best approximation to the time course through a particular voxel for the duration $\hat t$ of a stimulus.

So $\bf \hat Y$ is an $n \times \hat t$ matrix and therefore $\bf U$ is $n \times n$ while $\bf V$ is $\hat t \times \hat t$.

With respect to the second question. The equation given is

$\bf J = \bf U^T Y$

We are given that $\bf J$ is a 2 or 3 $\times t$ matrix. (This involves a small sleight of hand in dropping rows/columns.) Two or three is picked as the dimensionality as this is what can be plotted in figure 6 of the paper.

However $t \ne \hat t$ so I expect the separate traces (lines in fig 6) have been obtained by chopping $\bf J$ into the different segments corresponding to presentations of the stimulus. Each of these blocks can then be plotted in 2 or 3 dimensional space by considering each column as a point in that space and then drawing a line between the points defined by adjacent columns giving the trajectories.

Following on from the above video 8 appears for each block to add each (column-)point sequentially, join it to the last point, and render this length $\hat t$ sequence as a video.

I've not dealt with the colouring methodology before, and it would take a while before I was confident to comment on that aspect. I found the comment on similarity to Fig 4c confusing as the colouring is obtained there by per-voxel regression. Whereas in Fig 6 each trace is a whole-image artefact. Unless I'm put straight I think it's the direction of the stimulus during that time segment as per the comment in the Figure.

• The first figure above refers to an experiment with the same visual stimulus presented every time. There is a different figure and movie for those data. The second figure above refers to a different experiment in which the stimuli are visual stimuli with differing orientations, the traces in the 2nd figure above are colored to simply correspond to differing visual stimuli orientations. Nov 26, 2014 at 15:26
• Also, you are correct that the original vector $\mathbf Y$ is chopped up to lengths of $\hat {\mathbf T}$ $\n$ Nov 26, 2014 at 15:34
• You have confused me by discussing $\mathbf V$ and $\mathbf S$ in the equation $$\mathbf J = \mathbf U^\top \mathbf Y.$$ Do you mean the first 2 or 3 columns of $\mathbf U$? Nov 26, 2014 at 15:50
• I've re-arranged things. Apologies, was a left over from before I sorted something else out. Nov 26, 2014 at 16:07
• Thanks for all your help. Is the first principal component weights vector just the mean time series collapsing across all voxels? If it were the mean, it would result in the smallest scores to fit to the individual data traces. Nov 26, 2014 at 19:09