5
$\begingroup$

I am trying to better understand PCA as applied to time series by drawing parallels with this explanation of PCA as applied to images of faces. In particular, I would like to visualize the resulting "eigen-time series" in the case of PCA for time series in the same way that the explanation is able to visualize the eigenfaces.

My understanding of the image data is that each image is an array of (positive) integers. Performing PCA on the series of images requires transforming each 64x64 image into a 1x4096 row vector, where each of the 4096 columns corresponds to a specific pixel location. The resulting 4096 principal components are the eigenvectors, and depending on the number of principal components selected (say n_comp), some linear combination of the n_comp eigenvectors can be used to reconstruct the original face image (to some degree of accuracy).

Visualizing the original image involves mapping each integer in the 64x4 matrix to a gray scale color. As far as I can tell, although the eigenvectors are real numbers which are not necessarily positive, we are still able to visualize each of the n_comp "eigenfaces" by reshaping the 1x4096 vector into a 64x64 matrix, and mapping each real number in the 64x64 matrix to a gray scale color, because the ends of the color map are mapped to the minimum and maximum values in the 64x64 matrix.

Now suppose I have 250 time series each of length 96, and each time series is measuring the same thing just on 250 different days. Each column corresponds to the time series value at a specific time of day (e.g., the measurement taken at 1:15pm). After applying PCA, suppose that I find that the first 9 principal component explain 90% of the variance. Each principal component (eigenvector) is a 1x96 vector, and I would like to also visualize the principal component (eigenvector, "eigen-time series") in the same way that the eigenfaces were visualized. However, in my case the units of the vector do matter because the time series was measured in g/dL but the values of the principal component (as far as I can tell) have no units. Is there a way to transform the unitless principal component/eigenvector into an eigen-time series that does have the units of the original time series?

$\endgroup$
2
  • $\begingroup$ they're not unitless. imagine the 2d case if all your measurements are g/dl then any direction is also g/dl $\endgroup$
    – seanv507
    Commented May 26, 2020 at 21:25
  • $\begingroup$ I guess my issue is that, although the time series values must be non-negative (i.e., greater than or equal to 0), individual values of the eigenvector can be negative, so I am struggling to see how to 'convert' the eigenvector into the g/dL space. $\endgroup$ Commented May 27, 2020 at 0:33

2 Answers 2

1
$\begingroup$

Old post but replying for the benefit of progeny.

So I believe OP's confusion arises from how positive-valued measures can be recovered from derived negative eigen values. I believe what OP is overlooking is the step where the values are mean normalized before applying SVD.

"Going on, the next step is to define PCA. Its arguments are X which is vectors of images and n_pc which is the number of principal components. As a little reminder, principal components define an orthonormal basis that can extract the maximum variance in the original data. In our case, its shape is (1000, 4096) since we needed to transform the images into vectors for PCA. Then, we find the mean and subtract it from our data to center it around the origin. After that, we need to perform Singular Value Decomposition on the centered data to find those principle components called eigenfaces."

Basically, what was done to support the achieved compression (in non-technical terms) was to compute the "average face" and then subtract that average face from every individual face before computing the eigen-values/vectors via SVD. As such, these eigen-value/vectors represent the difference details from the "average face". I suspect that doing something similar to your time series example will allow you to reconstruct the series in a positive-definite manner (that is, take the "average series" and add the derived eigen values to compute the resultant values in higher-dimensional space). NOTE: I say "add" but really you are reconstructing the linear combination of the average representation with the derived eigen coefficients.

$\endgroup$
0
$\begingroup$

More on target is this work: 'Factor Analysis and Regression', to quote:

This paper derives a stochastic linear equation from factor analysis called factor analysis regression which is suggested as an alternative to classical least squares regression...

and as an extension, I would add, relating to the current question, also 'analysis of times series'.

Further comments include:

Factor analysis appears to be a particularly appropriate tool in the field of economics where many “independent” variables have high intercorrelation and where there are errors in all the variables.

And lastly:

Stochastic linear equations can be obtained from factor analysis which give better coefficients (better from the standpoint of their economic meaning and their theoretical expectation) then do regression equations obtained through a traditional least squares squares [9].

The downside is the magnitude of calculations and theoretical complexity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.