I have a PCA addin for MSExcel. One of the results it gives for each component is the R², cummulative R², Q², and cummulative Q². It looks like the cummulative R² is the sum of all previous R² values, while the cummulative Q² is:
$$ cummulativeQ^2_n = cummulativeQ^2_{n-1} +\left(1-cummulativeQ^2_{n-1}\right)*Q^2_n $$
Can anyone tell me what this Q statistic is, how it is calculated, and what it means?
For PCA $Q^2$ is a measure of the residual variation after applying the model to samples that have been held out, i.e. how much of a sample cannot be explained by the model. The difference with $R^2$ is that $R^2$ is used on the training set samples included in the current round of cross validation.
With each component you expect to fit progressively more of the variation (you do achieve this in included samples, but is not guaranteed in held out), so the $Q^2$ can decrease - you can see progressively higher residuals. This is why the cumulative function is not a sum of all previous. The metric has a range of 0 to 1 (see below for exception), so each component can only account for 1-previous $Q^2$, hence the $1−cumulativeQ^2_{n−1}$ term.
You need to calculate $Q_i^2$ for the $i^{th}$ PC directly, and the reference you give in the comments (http://wiki.eigenvector.com/index.php?title=T-Squared_Q_residuals_and_Contributions) does indeed indicate how this is done. Some others are
https://www.rdocumentation.org/packages/pcaMethods/versions/1.64.0/topics/Q2
https://umetrics.com/sites/default/files/books/sample_chapters/multimega_parti-3_0.pdf (Page 58)
It is calculated the same as $R^2$ but only on held out data.
The $Q^2$ is an indicator of how much variation is accounted for in your held out dataset. You hope to see this rise similarly to $R^2$, but as your model starts to overfit the two diverge more and more. More critically $Q^2$ is a very useful metric as it starts to drop again in held out samples if the model is overfitted. This indicates that the model PCs are accounting for variation that is not stable in the held out samples (which is the definition of an overfitted model). Lower order components of overfitted models contain non-reproducible variation, which actually then imputes this noise onto the residual after accounting for the model.
Like one version of $R^2$, it is possible to get a negative number indicating a very bad model that is adding noise to your held out samples When is R squared negative? http://forum.smartpls.com/viewtopic.php?t=15444
