Assessing Repeatability of a Multivariate Measurement

Question

I have a measurement that was taken 10 times on a sample. The measurement has roughly 11K points. If you're curious, I'm looking at mass spectrum data. I want to determine the repeatability of the spectra.

I am looking at either using ANOVA to do this, or using a PCA to determine how close the PC coordinates (PC1, PC2) are to each other. Would either of these options give a good measure of repeatability for this application or is there another method that would be better?

I've looked at this post and I've eliminated co-inertia analysis since it is used to compare only two multivariate data sets. Not sure how to use option 2 in the accepted answer and option 3 seems like using PCA.

Answer 1

Repeatability expresses how similar (or variable) measurements are when you try to keep conditions constant.

Expressing this in absolute numbers makes only sense if these absolute numbers can be interpreted. I suspect that a "repeatability of ± 700 ion counts at m/z 1000" or "average Euklidean distance between repeated measurements is 100 counts" doesn't help. Thus, you should probably relate the repeatability to something, e.g.

• the signal strength: that would get you to the classical signal to noise ratio (for the m/z 1000 peak, the SNR is 10000/700 ≈ 14".
• Or maybe relate the repeating variation to the variation you get due to some other influencing factors of interest. For this, ASCA (ANOVA simultaneous component analysis) is an extension/generalization of ANOVA that works well with highly multivariate data. Note that ASCA does first the ANOVA-like decomposition, and then PCA on the summand matrices.
  You can also look at between class : within class ratio of e.g. LDA (PCA-LDA for your highly multivariate data), or explained variance vs. residual (repetition) variance.

To get a start on ASCA, the Comprehensive Chemometrics have a nice chapter on it.

The Smilde group has a number of papers, e.g.
Smilde, A. K.; Hoefsloot, H. C. J. & Westerhuis, J. A. The geometry of ASCA Journal of Chemometrics, 2008, 464-471

I also found
Marini, F.; de Beer, D.; Joubert, E. & Walczak, B. Analysis of variance of designed chromatographic data sets: The analysis of variance-target projection approach Journal of Chromatography A, Elsevier BV, 2015, 1405, 94–102 and
Stanimirova, I.; Kazura, M.; de Beer, D.; Joubert, E.; Schulze, A.; Beelders, T.; de Villiers, A. & Walczak, B. High-dimensional nested analysis of variance to assess the effect of production season, quality grade and steam pasteurization on the phenolic composition of fermented rooibos herbal tea Talanta , 2013, 115, 590 - 599
helpful.

---

There's also an approach that first does a PCA and then an ANOVA. However, the ASCA people advise against that (and propose to work the other way round) as interesting structure correlating with the modeled factors may be easier to detect by ASCA. However, I guess if the goal is only to quantify repetition variance compared to other kinds of variance, first doing the PCA won't hurt.