1
$\begingroup$

First of all, I have to say that my knowledge of statistics is very basic. I was trying to fit data with a linear regression in Matlab, and I came across the problem of $R^2$ definition. I am using the free Ezyfit toolbox, but my question is about which definition of $R^2$ do I have to use in my analysis. I fitted the data both with a linear regression with intercept $y = ax + b$ and without intercept $y = ax $. I calculated $R^2$ from the definition $$ R^2 = 1 - \frac{SS_{res}}{SS_{tot}} $$

where $SS_{res}$ is the sum of squares of residuals and $SS_{tot}$ is the total sum of squares.

I saw that in the Ezyfit toolbox (and from what I can understand in the Matlab Statistics Toolbox as well, http://uk.mathworks.com/help/stats/coefficient-of-determination-r-squared.html) the definition used for $R^2$ instead is

$$ R^2 = \frac{SS_{reg}}{SS_{tot}} $$

where $SS_{reg}$ is the regression sum of squares.

From what I understand, the first definition is the most general one, and it is equivalent to the second definition only in some cases (when $SS_{tot} = SS_{reg} + SS_{res}$, see https://en.wikipedia.org/wiki/Coefficient_of_determination). When I calculated the $R^2$ for my linear regression with intercept, the two definitions gave the same result. But when I calculated it for the regression with intercept 0, the results of the two $R^2$ definitions differ. And in my case, if I use the first definition the regression with the intercept is the best fit, with a higher $R^2$, and if I use the second one the 0 intercept is the best fit. If I use Ezyfit, that finds the regression parameters by minimising $SS_{reg}$, it gives me an intercept significantly different from 0, even if the $R^2$ of the 0 intercept is higher according to the definition used inside the toolbox.

I am interested in understanding why Matlab and Ezyfit use the second definition, and in what cases in general it is possible or preferable to use it. In my particular case, which of the two definitions is the more appropriate? Thank you in advance


Edited: I think my question is different than Removal of statistically significant intercept term increases $R^2$ in linear model, because there it's R that uses a modified version of $R^2$ if you don't set the intercept, while here I am calculating the $R^2$ myself. From my results, the $R^2$ definitions are equivalent for a fit with intercept, but different without intercept, and I would like to understand why.


Edited (with answer): Thanks everyone. The core of my problem was that the two versions of $R^2$ calculated from me had different results in the case of a linear regression without intercept. This was my mistake, as the second version of $R^2$ is not applicable for a fit without intercept. Now I have a different problem, that I'm going to ask in a different question

$\endgroup$
  • 1
    $\begingroup$ How are the SS calculated in these two cases. Typically, there is a "corrected total" which is used when you have the intercept. I don't know what MATLAB does. What happens when you calculate the SS manually / naively (& equally) for both models? $\endgroup$ – gung - Reinstate Monica Oct 29 '15 at 15:52
  • 1
    $\begingroup$ I think if you were to refocus your question on "To what extent are the different versions of $R^2$ comparable, particularly without an intercept" you might be able to create a separate question from that. If you look at the bottom of @cardinal's answer here, you can see Andy Clifton wrote "'I'm missing something. Is what R does, correct? I mean is the R^2 value that is reported, even remotely comparable between the with and without intercept cases?" This seems to be the crux of your question too. $\endgroup$ – Silverfish Oct 29 '15 at 18:35
  • 1
    $\begingroup$ If this is the angle you are going for, you might want to severely edit this question (I'd recommend to completely de-emphasise or even remove all implementation-specific issues re Matlab or your toolbox) or post a fresh one. It would be considered inappropriate to post a new question rather than editing your old one if your new question is intended as a "fix" for your old one, but in this case it seems you have established an answer to your original question from one of the linked threads, and now have a follow-up that does not sound like a duplicate anymore. $\endgroup$ – Silverfish Oct 29 '15 at 18:36
  • 1
    $\begingroup$ (Regardless of whether you post a fresh question, or edit this one, you should make it clear what you have read and understood so far, and explain why your question is asking something new that wasn't covered there. Otherwise there is a chance of it being closed as a duplicate, or of getting answers that simply go over the same ground as you already know.) $\endgroup$ – Silverfish Oct 29 '15 at 18:39
  • 1
    $\begingroup$ Thank you very much, @Silverfish. I'll write a new question and edit this one, to explain what I have understood. I hope I'm doing it in the right way, thank you again. $\endgroup$ – Letty Oct 30 '15 at 10:05

Browse other questions tagged or ask your own question.