I am following a collaborator’s methods to analyze a set of audio recordings, and I have found that she is using principal components analysis in an unexpected way. I am confused by her approach, and would appreciate feedback on whether this is sound way to think about and use PCA.

I have several audio recordings each from different speakers and the ultimate goal is to use a canonical variates analysis to assign these recordings to the correct speaker based on a reduced set of variables. To obtain these variables, I divide each recording into 80 frames and take 20 different measurements for each frame. (Full disclosure, these 20 variables are mel-frequency cepstral coefficients, but you can think of them as typical acoustic parameters like pitch and amplitude to make this clearer.) This leaves me with 80×20 measurements for each recording, and I want to reduce these to a smaller set of variables that I will actually use to compare distinct recordings in my future canonical variates analysis. To me, using PCA seems like a reasonable approach here, and my collaborator agrees. However, what she does, under the title “PCA”, is to calculate the eigenvalue of each variable across the 80 frames, and retain this variance value. Since there is only one variable, there is only one principle component, and only one variance value (which explains 100% of the variance). Therefore she is left with her same 20 variables, but with the variance across frames as the value for each variable. She then proceeds to reduce these 20 variables through canonical variates analysis (one could also imagine reducing them with another PCA, without classification).

I am 1) confused as to why this is termed PCA when all she is doing is taking the variance of her measurements and 2) unsure if this is a valid approach to reducing the number of variables in the dataset. This does not fit with how I typically use PCA, and I’m unsure if that’s simply my inexperience or because this is an inappropriate use of PCA.

My collaborator is senior and extremely busy. I can ask her these questions directly, but wanted to exhaust all other options first, including appealing to the wider stats universe. What am I not getting here?

Edit: There is a paper published with this method, but the details of why the authors reduced the multiple frames to one variance measurement are not provided and this step is not mentioned (although by looking at the data it is clear that this is what happened).

Performing a PCA on the time series data could be another useful approach. However, I am not sure it makes sense to use data for all of my speakers to reduce dimensions, since ultimately I am looking for consistent differences between speakers, whose variables may be substructured in different ways. My dataset consists of multiple recordings from multiple speakers – in the canonical variates analysis, I will be grouping according to speaker.

  • $\begingroup$ Is there a paper than can be cited or uses a similar approach? A standard approach would be to get $N$ ceptrums ($N$ being the number of users), perform dimensionality reduction to these ceptrums (eg. FPCA as to account for the time-varying ceptrum) and feed the PC scores into a classifier. One needs to use the recordings from the $N$ speakers together! This appears as a within-subject normalisation step first (from what I can tell, apologies if I misunderstood something) that probably is application specific. Can you please give more details about your goal (and define what CVA stands for)? $\endgroup$
    – usεr11852
    Jan 28, 2019 at 23:19
  • $\begingroup$ CVA stands for canonical variates analysis, or discriminant function analysis. I've addressed your other questions and suggestions in an edit in the original question. $\endgroup$
    – Amanda S
    Jan 29, 2019 at 23:07
  • $\begingroup$ Calculating statistical summarizations of MFCC coefficients (along the time axis) and then using this as features is quite common. This reduces from n_mfccs*n_time_frames to n_mfccs. mean+std is a common one, but also max+min, and sometimes skew/kurtosis. $\endgroup$
    – Jon Nordby
    Apr 12, 2019 at 21:00
  • $\begingroup$ I did however not understand your description of the "PCA" method in use. I got lost at the part about "since there is only one variable" $\endgroup$
    – Jon Nordby
    Apr 12, 2019 at 21:02

1 Answer 1


Summarizing MFCC frames (n_mfcc, n_timesteps) for an entire clip is done quite often (at least pre deep-learning). The most common summarization is to take the mean and standard deviation across time (for each MFCC coefficient). The result is a feature vector (2, n_mfcc) - often flattened to a 1D vector (2xn_mfcc,). Then any ML classifier can be used, such as LogisticRegression or RandomForest.

This however has nothing to do with PCA.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.