PCA with on data with unequal length

I am wondering how to perform PCA on data with unequal lengths. In the simplest case you could split up the data matrix so that one block would be $T_{1}Xn$ and the rest is $T_{2}Xn$. The second block has effectively missing data in the first $T_{1}-T_{2}Xn$ block.

Many of the applications of PCA on missing data are variants of the EM algorithm methodology (these typically will assume the distribution is multivariate Gaussian) that are more suited to applications where the data is missing at random. Oftentimes, these techniques do not particularly apply to data with uneven lengths. However, there are techniques to estimate the mean and covariance matrix in these cases and it is possible to apply PCA to the resulting covariance matrix. However, when the number of variables becomes large (absolutely or relative to the number of observations), then it may no longer be convenient to estimate the covariance matrix in this fashion.

• Excuse me, what does this mean data is T1Xn and T2Xn? And did by "unequal length" you just mean that there are missing values? Commented Jul 5, 2012 at 19:24
• So the data is $TXn$, but you could break it into the respective panels that would be of different length. The first $T_{2}-T_{1}Xn$ block would be missing in the second panel.
– John
Commented Jul 5, 2012 at 19:45
• I'm sorry if this comes a bit too late... I'm currently looking for a solution to the same problem. You mention that there are techniques to estimate the mean and covariance matrix in these cases and it is possible to apply PCA to the resulting covariance matrix... would you mind kindly giving me some references? Thank you in advance! Commented Mar 18, 2021 at 15:56
• This is a common one: ideas.repec.org/p/fth/pennfi/05-96.html
– John
Commented Mar 18, 2021 at 22:10

What I've seen is an interpolation of the data in order to match the length (and even the sampling) of the data. This was particulary used (successfully, I must say) in the classification of variable stars using PCA, where the data where the actual light curves of the stars. For more details, see the paper of Deb & Singh (2009).

I must add, however, that if you are going to do PCA interpolating data, maybe Functional Data Analysis techniques are more suitable (I've been thinking in writing a paper with this method, in fact, and compare it with the PCA approach).

• It looks to me like that paper interpolates missing data within the sample. Hence, that might be a situation where one could apply the EM algorithm approach. I am thinking about the situation where there is a lot of missing data at the beginning of some series.
– John
Commented Jul 5, 2012 at 19:51
• I'm also not that familiar with Functional Data Analysis, so I'm not sure how to go about applying that.
– John
Commented Jul 5, 2012 at 19:52
• If there is not much data at the beggining, then extrapolating could be a solution but, as you say, there's a lot of data missing so I don't really know if this approach would work for you then. FDA is just as PCA applied to continuous functions (hence the name). For an introduction to the subject, you should check out Ramsey & Silverman's book: springer.com/statistics/statistical+theory+and+methods/book/…. Commented Jul 5, 2012 at 20:09
• Extrapolating is like the technique used to estimate the mean and covariance matrix for unequal data. Basically, you would estimate the mean and covariance of the data that is longest and then regress the smaller dataset on the longer dataset. The error estimates would be calculated over the shorter period, but you could use the whole series of the longer dataset to predict the values of the shorter series at the longer lengths.
– John
Commented Jul 5, 2012 at 20:15

PCA is not the only dimensionality reduction that can be used, there are plenty of other methods, e.g. you may consider feature extraction to extract distinctive features from your data (with different lengths). FFT on a sliding window; overlapped or not, might be a good option, DCT of the results of FFT or without might be another option, it all depends on the Data used.