Timeline for Removal of statistically significant intercept term increases $R^2$ in linear model
Current License: CC BY-SA 3.0
20 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 22, 2022 at 21:33 | answer | added | user4422 | timeline score: 4 | |
| Apr 28, 2017 at 16:28 | answer | added | Jonathan Harris | timeline score: 2 | |
| Dec 27, 2015 at 14:43 | comment | added | user83346 | see this link for an explanation: stats.stackexchange.com/questions/171240/… | |
| Dec 27, 2015 at 14:07 | history | edited | Ernest A | CC BY-SA 3.0 |
changed wording in title
|
| Sep 26, 2015 at 20:31 | history | edited | gung - Reinstate Monica |
changed tag
|
|
| Apr 23, 2012 at 18:59 | history | edited | cardinal |
edited tags
|
|
| Apr 11, 2012 at 23:38 | history | tweeted | twitter.com/#!/StackStats/status/190222438154510337 | ||
| Apr 11, 2012 at 2:59 | history | edited | cardinal | CC BY-SA 3.0 |
Added r tag. Tweaked title.
|
| Apr 10, 2012 at 17:31 | vote | accept | Ernest A | ||
| Apr 10, 2012 at 16:41 | answer | added | cardinal | timeline score: 189 | |
| Apr 10, 2012 at 14:50 | comment | added | Ernest A | @cardinal: the end goal is to express $\alpha$ as a function of $\delta$. To put some perspective, $\alpha_i$ and $\delta_i$ are maximum likelihood estimators from another model (call it model A). What I want is reduce the number of parameters of model A. So if I can say $\alpha = k \delta$, for example, then I can change the specification of model A and cut the number of parameters in half (minus one), which is what I want. | |
| Apr 10, 2012 at 14:36 | comment | added | cardinal | The $R^2$ is not necessarily larger. It's only larger without an intercept as long as the MSE of the fit in both cases are similar. But, note that as @Macro pointed out, the numerator also gets larger in the case with no intercept so it depends on which one wins out! You're correct that they shouldn't be compared to one another but you also know that the SSE with intercept will always be smaller than the SSE without intercept. This is part of the problem with using in-sample measures for regression diagnostics. What is your end goal for the use of this model? | |
| Apr 10, 2012 at 14:28 | comment | added | Ernest A | @cardinal: Makes sense. So, to summarize, without intercept the $R^2$ is larger but it's not comparable to the $R^2$ of the model with intercept. Instead, the sum of squared residuals can be used to compare the goodness of fit of both models. Correct? | |
| Apr 10, 2012 at 14:07 | comment | added | cardinal | What $R$ does when there is no intercept is that it calculates $$R^2 = 1 - \frac{\sum_i (y_i - \hat y_i)^2}{\sum_i y_i^2}$$ instead (notice, no subtraction of the mean in the denominator terms). This makes the denominator larger which, for the same or similar MSE causes the $R^2$ to increase. | |
| Apr 10, 2012 at 14:04 | comment | added | cardinal | Related: stats.stackexchange.com/questions/7357 | |
| Apr 10, 2012 at 13:11 | comment | added | Macro | Well, the RSS has to go down (or at least not increase) when you include an additional parameter. More importantly, much of the standard inference in linear models does not apply when you suppress the intercept (even if it's not statistically significant). | |
| Apr 10, 2012 at 12:31 | comment | added | Ernest A | @Momo: Good point. I've calculated the residual sums of squares for each model, which seem to suggest that the model with intercept term is a better fit regardless of what $R^2$ says. | |
| Apr 10, 2012 at 12:29 | history | edited | Ernest A | CC BY-SA 3.0 |
added RSS
|
| Apr 10, 2012 at 11:41 | comment | added | Momo | I recall $R^2$ to be the ratio of explained to total variance ONLY if the intercept is included. Otherwise it can't be derived and loses its interpretation. | |
| Apr 10, 2012 at 11:29 | history | asked | Ernest A | CC BY-SA 3.0 |