0
$\begingroup$

I have two models, one lm(y ~ x1 + x2 + 0) which gives me a close to 0.90 something $R^2$ and another model lm(y ~ x1 + x2) which gives me a close to 0.0003 ish $R^2$. Based on $R^2$ one might choose the first model. I have domain (prior) knowledge that a 0 value of $x_1$ and a 0 value of $x_2$ should produce a 0 value of $y$. So, is my explicit setting of 0 a fair thing? Based on this knowledge and the obtained value of $R^2$ can I go with the first model?

Edit: From the threads Removal of statistically significant intercept term increases $R^2$ in linear model and When is it ok to remove the intercept in a linear regression model? I have learned that it is okay to do it as long as the domain knowledge above holds true. From Is $R^2$ useful or dangerous? I learned that $R^2$ alone is insufficient to make a decision about how good the model is. I need to look at AIC and BIC to make further decisions. However, because I my background is not in statistics, I am having trouble understanding the following in the first link:

  1. Why does the first equality occur only when intercept is included?
  2. why does $\bar{y}$ become 0 when intercept disappears?
  3. Why in the second case, $R_0^2$ uses a reference model corresponding to noise only?
  4. What is the $\bar{y_1}$ in the notation?
  5. Why is $\|\ y \|_2^2 = \| y - \bar{y_1}\|_2^2 + n \bar y^2$ true?
$\endgroup$

marked as duplicate by gung regression Nov 27 '14 at 18:44

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 2
    $\begingroup$ Suppressing the intercept is almost never recommended (see: When is it OK to remove the intercept in lm()). In addition, the $R^2$ values are not comparable (see: Removal of statistically significant intercept term boosts $R^2$ in linear model). $\endgroup$ – gung Nov 27 '14 at 18:43
  • $\begingroup$ I think you will learn what you need in these threads. Please read through them. If you still have a question afterwards, come back here & edit your Q to state what you have learned & what you still need to understand, & we can re-open this & provide the information you need, rather than just duplicating material found elsewhere that wasn't enough for you. $\endgroup$ – gung Nov 27 '14 at 18:46
1
$\begingroup$

In R formula +0 is the same as -1 and it means that you want a model with no intercept. Also, if regression model is:

$$y = \alpha + \beta_1 x_1 + \beta_2 x_2$$

so if $x_1 = x_2 = 0$, then:

$$y = \alpha + \beta_1 \times 0 + \beta_2 \times 0 = \alpha$$

so if you know that if $x_1 = x_2 = 0$ then $y=0$, it also means that in this case $\alpha$ has to be $0$.

On another hand, removing an intercept in many cases is not a good idea (as also gung suggeses in a comment). In your case model with no intercept will provide a perfect fit for values only in the case where $x_1 = x_2 = y = 0$, but could give worse fitting in other cases so you should consider if this idea is resonable.

However, you do not "explicitly set 0" in here, but remove an intercept (i.e. $\alpha$) from the model. $R^2$ also suggests that the model without intercept is better, however beware that $R^2$ values could be misleading and you should not judge a model only basing on $R^2$.

$\endgroup$
  • $\begingroup$ I'm confused. what do you mean by "do not "explicitly set 0" in here, but remove an intercept " So, the way you suppress an intercept in R is by adding a +0 (or a -1) to the model right? $\endgroup$ – rk567 Nov 27 '14 at 21:20
  • 1
    $\begingroup$ Yes, you remove intercept this way. If you meant that removing intercept is setting it to 0 than it seems I just have misunderstood you. $\endgroup$ – Tim Nov 27 '14 at 21:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.