3
$\begingroup$

I saw this answer (https://stats.stackexchange.com/a/6374/27969) that said that $R^2$ can be negative if it's defined as $1-\frac{SSR}{SST}$. SSR is the sum of squared residuals which is defined as

\begin{align} SSR &= \displaystyle\sum\limits_{i=1}^n (y_i - \hat{y}_{i})^2 \\ &= \displaystyle\sum\limits_{i=1}^n (y_i - \hat{\beta_1} x_{i1} - \hat{\beta_2} x_{i2} - \ldots - \hat{\beta_k} x_{ik} )^2 \\ \end{align}

when theres no intercept, and SST is the total sum of squares, or

\begin{align} SST &= \displaystyle\sum\limits_{i=1}^n (y_i - \bar{y})^2 \\ \end{align}

Also SSE, or the explained sum of squares, is

\begin{align} SSE &= \displaystyle\sum\limits_{i=1}^n (\bar{y} - \hat{y}_i)^2 \end{align}

in the notation of my book. I just got to the section in my textbook that mentions this too and I'm confused. Why is this true?

Here's what I could work out. I know that

\begin{align} R^2 &= 1-\frac{SSR}{SST} \\ \end{align}

So for $R^2 < 0$, we need $\frac{SSR}{SST} > 1$. Here's what I worked out:

\begin{align} SSR &> SST \\ \displaystyle\sum\limits_{i=1}^n (y_i - \hat{y}_{i})^2 &> \displaystyle\sum\limits_{i=1}^n (y_i - \bar{y}_{i})^2 \\ \end{align}

Here's the proof I tried to work out:

\begin{align} SSR &> SST \\ SSR &> \displaystyle\sum\limits_{i=1}^n (y_i - \bar{y})^2 \\ &> \displaystyle\sum\limits_{i=1}^n \left[ (y_i - \hat{y}_i) - (\bar{y} - \hat{y}_i)\right]^2 \\ &> \displaystyle\sum\limits_{i=1}^n (y_i - \hat{y}_i)^2 - 2\displaystyle\sum\limits_{i=1}^n (y_i - \hat{y}_i)(\bar{y} - \hat{y}_i) - \displaystyle\sum\limits_{i=1}^n (\bar{y}_i - \hat{y}_i)^2 \\ &> SSR - 2\bar{y}\displaystyle\sum\limits_{i=1}^n (y_i - \hat{y}_i) + 2\displaystyle\sum\limits_{i=1}^n \hat{y}_i(y_i - \hat{y}_i) - \displaystyle\sum\limits_{i=1}^n (\bar{y} - \hat{y}_i)^2 \\ &> SSR - 2\bar{y}\displaystyle\sum\limits_{i=1}^n (y_i - \hat{y}_i) + 2\displaystyle\sum\limits_{i=1}^n \hat{y}_i(y_i - \hat{y}_i) - SSE \\ 0 &> -2\bar{y}\displaystyle\sum\limits_{i=1}^n (y_i - \hat{y}_i) + 2\displaystyle\sum\limits_{i=1}^n \hat{y}_i(y_i - \hat{y}_i) - SSE \\ SSE &> -2\bar{y}\displaystyle\sum\limits_{i=1}^n (y_i - \hat{y}_i) + 2\displaystyle\sum\limits_{i=1}^n \hat{y}_i(y_i - \hat{y}_i) \\ \end{align}

This is where I get stuck. Is the proof on the right path, or is there a mistake before I got stuck? Hints would be great.

I'm starting to get why this is the case logically. SSR is the sum of the squares of the distances from the points to the regression line, while SST is the sum of the squares of the distances from the points to a line drawn through $\bar{y}$. $R^2$ is negative because $SSR > SST$, which means that because we impose the limitation that the regression line MUST pass through the origin, our regression line is an even worse fit than if we just drew a horizontal line at $y =\bar{y}$ and used that. The distances from the points to the regression line are higher than the distances from the points to the line at $\bar{y}$ (ok thhe sum of squares of the distances is larger). Is that the right logic?

$\endgroup$
  • 1
    $\begingroup$ There are other discussions about the R-squared when there's no interecept, such as stats.stackexchange.com/questions/47527/… Did you take a look at these discussions too ? $\endgroup$ – Stéphane Laurent Jul 27 '13 at 19:27
  • $\begingroup$ @StéphaneLaurent I guess I didn't understand why SST is the sum of squares around 0 in the model with no intercept (the other answer says that), since it's still defined the same way. I think my question is actually a stupid question because I won't be able to prove mathematically that $SSR > SST$ because even though it might be true, it's not always true. Is that right? $\endgroup$ – M T Jul 27 '13 at 19:34
  • 1
    $\begingroup$ To prove something can happen, you need only find an example, don't you? $\endgroup$ – Glen_b Jul 28 '13 at 0:53
  • $\begingroup$ @Glen_b yeah this was a stupid question I realize now. what do I do just delete it? $\endgroup$ – M T Jul 28 '13 at 1:31
  • $\begingroup$ @Michael I think the title question is a useful question; some of the body might need to change though. If you think you can answer the question (from the sound of it I think you could), you should do so. If not, you can remove the parts that seek to prove something you don't need to and leave it to be answered; the question should stay $\endgroup$ – Glen_b Jul 28 '13 at 14:07
2
$\begingroup$

As you said, the problem is that it is not always true that $R^2<0$ when there is no intercept and therefore you just need to find and example to show that $R^2$ may be less than 0.

I'm just proposing an extreme example where $SSR>SST$:

Residuals for y=1

In any sample with y=constant and $\bar x$ different than 0, $SSR>0$ (the squares of the blue lines in the graph) while $SST=0$. Please notice that if we considered an intercept, the regression line would be horizontal and $SSR$ would equal zero, too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.