Negative R squared contradicts ssa/sst?

Question

I understand from this question - When is R squared negative?

that the R squared value of a linear regression model can be negative if the intercept is constrained. And this makes sense if you define R squared as -

$$R^2 = 1-\frac{SSE}{SST}$$

One says $SSE>SST$. But then, $$SST = SSA + SSE $$ Total Sum of squares = Sum of squared errors + Sum of squared residual. And with this we get - $$R^2 = \frac{SSA}{SST}$$ And now it is hard to imagine how $R^2$ can be negative. Aren't SSA and SST >0 always?

Answer 1

$$ \begin{eqnarray*} SST&=&\sum_{i=1}^n (y_i-\bar y)^2\\&=&\sum_{i=1}^n (y_i-\hat y_i+\hat y_i-\bar y)^2\\&=&\sum_{i=1}^n (y_i-\hat y_i)^2+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)+\sum_{i=1}^n(\hat y_i-\bar y)^2\\&=&SSE+SSR+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y) \end{eqnarray*} $$ When an intercept($\beta_0$) is included in linear regression, $$\hat{y}_i = \beta_0 + \beta_1x_{i,1} + \beta_2x_{i,2} +…+ \beta_px_{i,p}$$ It can be proved that $\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)=0$ here

So $$ SST=SSE+SSR+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)=SSE+SSR $$

$$ R^2 = 1-\frac{SSE}{SST}=\frac{SSR}{SST}\geqslant0 $$ However, when an intercept is not included in linear regression, $\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)\neq0$

If $\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)<0$, $SST=SSE+SSR+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)$may be less than $SSE$

In this case, $$ \frac{SSE}{SST}>1 $$ $$ R^2 = 1-\frac{SSE}{SST}<0 $$

Answer 2

The point in the accepted answer to your linked question illustrates the answer to this.

If the regression model is so bad (for example because of a foolishly constrained intercept, but there are other potential causes) that a constant predictor (i.e. a horizontal line, with an intercept equal to the mean of the observed data) would fit the observed data better, then the sum of squares of the errors will exceed the sum of squares calculated from the original data (of the difference from the mean), and you will find $R^2$ going negative.

In such a case, you might want to think about whether the expression $SST = SSA + SSE$ is meaningful when the model is so bad that $SSE \gt SST$ and $SST$ represents the sum of squares calculated from the original data.

Unconstrained simple linear regression avoids this problem, since it just minimises $SSE$, and so $SST$ is an upper bound on $SSE$

Answer 3

I found a proof for SST=SSE+SSR in a different thread (link below). The proof relies on two equations derived from least square regression. If your regression was not not based on LS, those conditions would be violated and the equation SST=SSE+SSR wouldn't hold, which removes the contradiction of a negative R-squared.

https://math.stackexchange.com/questions/709419/prove-sst-ssessr