# Does PCA have any advantages or usages in the frequency domain?

My question is about analysis of signals with PCA in the frequency domain.

• As frequency analysis offers a powerful tool for signal processing, does Principal Component Analysis (PCA) have any "well-known" advantages or usages in the frequency domain?
• Can it be stated that PCA is doing a kind of "frequency analysis" as the principal components represent the most important frequencies in the signal?
• What do you have in mind in applying PCA to signal processing? Let me elaborate. You refer to "the signal," suggesting there's only one. What would be the corresponding multivariate dataset of multiple observations to which you would apply PCA?
– whuber
Commented Oct 3, 2011 at 23:35
• (you might have mislead with my "not so good" English) I will try to further explain myself: I have several (many) samples of signal which comes from two distinct groups. my aim is to classify those two types of signals. Because the data is high dimensional one, I use PCA to extract scores for each signal. In principle/"theory" if I first transferred my signal to the time domain and then perform PCA, does it have any advantages ? this is what I meant when I asked "does (PCA) have any "well-known" advantages or usages in the frequency domain?"
– Dov
Commented Oct 4, 2011 at 6:06

The advantage of using PCA in the frequency domain is to choose a set of weights by exploiting the cross-correlations between the signals at particular cycles. For example, (depending on the field of application) the behaviour of the variables under study can be different in the short, medium and long run. Using PCA in the frequency domain will allow to choose weights depending on the frequency.

The difference between PCA in the time domain and frequency domain can be understood in terms of how the eigenvalues are computed. In time domain, the correlation matrix is used. In the frequency domain, the fourier transform of the correlation matrix or the spectral density matrix is used to compute the eigenvalues.

For technical applications of using PCA in frequency domain, there is a description in book by Jolliffe,I.T(2002), Principal Component Analysis, 2nd Edition. Here is a link to the relevant page.

Regarding your second question, I have understood PCA by itself to be a method of finding combinations of variables which extract the maximum information in the data by maximizing the variance of the principal components. Therefore, it does not seem to be dealing with any cyclic or frequency information in the data.

• thanks! this is what I look for! & I found more info at "Brillinger, 1981, Section 9.5" as Jolliffe cited him "There is a connection between frequency domain PCs and PCs deﬁned in the time domain"
– Dov
Commented Oct 15, 2011 at 19:41

There was a paper at the recent ICML by Li and Prakash. It is about a complex linear dynamical system, which turns out to be a model of which Fourier transform and PCA are special cases. Have a look at Time Series Cluster: complex is simpler.

Principal component analysis (PCA) is a mathematical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of uncorrelated variables called principal components. This transformation is defined in such a way that the first principal component has as high a variance as possible (that is, accounts for as much of the variability in the data as possible), and each succeeding component in turn has the highest variance possible under the constraint that it be orthogonal to (uncorrelated with) the preceding components. Principal components are guaranteed to be independent only if the data set is jointly normally distributed. I am not much knowledge about frequency analysis or signal processing. But PCA has application in this field. Check literature.

• Thanks for your quick answer! would love to know more if someone have experience with frequency analysis or signal processing in the context of PCA ?
– Dov
Commented Oct 3, 2011 at 23:30
• The way you ask the question, I think you dont have clear idea what is PCA. Could you please tell me what is your objective? Commented Oct 3, 2011 at 23:39
• (you might have mislead with my "not so good" English) I will try to further explain myself: I have several (many) samples of signal which comes from two distinct groups. my aim is to classify those two types of signals. Because the data is high dimensional one, I use PCA to extract scores for each signal. In principle/"theory" if I first transferred my signal to the time domain and then perform PCA, does it have any advantages ? this is what I meant when I asked "does (PCA) have any "well-known" advantages or usages in the frequency domain?"
– Dov
Commented Oct 4, 2011 at 6:05

If you accept spectroscopy as frequency-domain: PCA is used a lot there. E.g. a pubmed search on principal component analysis and spectroscopy yields more than 2500 results. On the other hand, spectroscopists rarely look at the time domain (Fourier-transform spectroscopy does use a spatial domain as "intermediate" but for data analysis the frequency/wavelength/wavenumber domain is generally used and interpreted).

If you do a PCA in the frequency domain, the first PCs will tell you which frequencies contribute most to the variance in the data set and moreover which frequencies vary together (with pos. or neg. correlation): they end up in the same PC or independent of each other (end up in different PCs). Whether this coincides with most contributing frequencies depends on whether/where your data is centered (by its nature or by centering).

I'd say whether PCA should be done in time or frequency domain depends on the interpretation of these domains.

• If you want to find things that happen at the same time or times where the same things happen, then the time domain should be appropriate.
• IF you want to find things that happen with the same frequency or frequencies where the same things happen, then use frequency domain.

PCA, like wave transformations, is an orthogonal transformation. It tries to rotate data to maximize certain metric (i.e. the variance). Interestingly, if the data is sufficiently random (isotropic), no rotation will help to enlarge variance. In this case, PCA will basically turn to a wave transformation.

• thanks! I'll be appreciated if you can point me to a reference ?
– Dov
Commented Oct 5, 2011 at 19:44