# Validation metrics (R2 and Q2) for Partial Least Squares (PLS) Regression

I'm attempting to validate my Partial Least Squares (PLS) -regression model. From documentation and other readings regarding PLS regression I've come to understand that there are generally two metrics used to evaluate the performance of the algorithm. $R^2$ is calculated as 1 - residual sum of squares(RSS) and the total sum of squares(TSS):

$$R^2 = 1 - RSS/TSS$$ $$RSS =\sum(y-\hat{\mathbf{y}})^2$$ $$\ TSS = \sum(y - \bar{\mathbf{y}})^2$$ While $Q^2$ is calculated as 1 - Predictive residual Error sum of squares(PRESS)/ TSS: $$\ Q^2 = 1 - PRESS/TSS$$ $$\ PRESS = \sum(y-\hat{\mathbf{y}})^2$$

The Calculation for $R^2$ and $Q^2$ are almost identical, with the only difference being that RSS is calculated from the data on which the algorithm is trained and PRESS is calculated from held out data.

My question:

In the view of training/test splits of data, is it appropriate to call $R^2$ a metric of how the algorithm fits the training data and $Q^2$ a metric of algorithm performance on test data?

Side question: Is it good practice to scale Y in the same manner as X in PLS regression?

• I think you should reword your question so that it is more about PLS and its performance measures rather than specifically about sklearn functions. I also think the score returning functions are very likely irrelevant of y predictions. Thus, it is probably not the correct way of calculating $Q^2$ and $R^2$. Scaling Y, on the other hand, depends on your data and if you don't know what makes more sense, you can always test different scaling and centering scenarios. – theGD Jul 21 '17 at 16:28
• @theGD I have changed the question to be more specifically about PLS-R performance measures. – Tkanno Jul 25 '17 at 12:41

In general, I think you have the right idea. According to Eriksson et al, the fit tells us how well we are able to mathematically reproduce the data of the training set. The $$R^2$$ parameter is known as the "goodness of fit", or explained variation. The $$Q^2$$ parameter is termed "goodness of prediction", or predicted variation.
• In PLS, the terms $$R^2$$ and $$Q^2$$ generally refer to the model performance of the Y-data, the responses, rather than that of the X-data, the predictors.
• The two parameters vary differently with increasing model complexity. $$R^2$$ is inflationary and rapidly approaches unity as model complexity (number of model parameters) increases. Therefore, it is not sufficient only to have a high $$R^2$$. $$Q^2$$, on the other hand, is not inflationary and at a certain degree of complexity will not improve any further and then degrade.