# What is "explained" by the explained/regression sum of squares?

We are in a regression setting. Let's start by defining some notation and terminology.

$$y_i$$ is observation $$i$$ of some (response) variable $$Y$$.

$$\hat{y}_i$$ is the value of $$y_i$$ predicted by a regression.

$$\bar{y}$$ is the average of all observations of $$Y$$.

$$y_i-\bar{y} = (y_i - \hat{y_i} + \hat{y_i} - \bar{y}) = (y_i - \hat{y_i}) + (\hat{y_i} - \bar{y})$$

$$( y_i-\bar{y})^2 = \Big[ (y_i - \hat{y_i}) + (\hat{y_i} - \bar{y}) \Big]^2 = (y_i - \hat{y_i})^2 + (\hat{y_i} - \bar{y})^2 + 2(y_i - \hat{y_i})(\hat{y_i} - \bar{y})$$

$$SSTotal := \sum_i ( y_i-\bar{y})^2 = \sum_i(y_i - \hat{y_i})^2 + \sum_i(\hat{y_i} - \bar{y})^2 + 2\sum_i\Big[ (y_i - \hat{y_i})(\hat{y_i} - \bar{y}) \Big]$$

$$SSRes := \sum_i(y_i - \hat{y_i})^2$$$$SSReg := \sum_i(\hat{y_i} - \bar{y})^2$$$$Other = 2\sum_i\Big[ (y_i - \hat{y_i})(\hat{y_i} - \bar{y}) \Big]$$

The interpretation of the $$SSRes$$ seems straightforward enough, just the sum of the squared differences between the predicted and the true values. Why we would square these instead of taking the absolute value is not immediately obvious, but it at least makes sense why we would care about the difference between the true and predicted values.

What intuition is there for $$SSReg?$$ Why should we care about the distance between the predicted values and the average value? Further, what does this have to do with an "explained" sum of squares? What is being explained?

I can wrap my head around this when $$Other = 0$$, such as in OLS linear regression, since the $$SSReg$$ and $$SSRes$$ add up to the total sum of squares. When $$Other \ne 0$$, however, I am unsure how to interpret the $$SSTotal$$ decomposition beyond the $$SSRes$$.

• Re "Why should we care about the distance between the predicted values and the average value?" Because that's the simplest possible OLS model (based on the data, anyway), and thereby serves as a unique frame of reference. When the mean predicts the data adequately, you aren't obliged to complicate your analysis by throwing in more variables.
– whuber
Commented Jul 5, 2023 at 14:11
• If you turn classic books of yore, they always mentioned it as the reduction in sum of squares. That is the amount of sum of squares that could reduced from the total sum of squares "attributable to having fitted the regression". Commented Jul 5, 2023 at 14:39
• @User1865345-solidarityMods I buy that for OLS linear regression, but what about when $Other \ne 0?$
– Dave
Commented Jul 5, 2023 at 16:56

• In what way does $SSRes$ consider what has been accounted for? Just because it’s not the residual $SSRes?$ But then what about that “other” term in the decomposition of the total sum of squares?
• $Other=0$ is the exception, not the rule. You will get $Other\ne 0$ in ridge or LASSO regression, for instance.