1
$\begingroup$

I'm comparing a response variable to multiple explanatory variables using a multiple variate linear model (lm() in R). My metric for a good fit is currently $R^2$ (or adjusted $R^2$) and RSE. At times, it generates a non-statistically significant intercept. I know removing it distorts my $R^2$ and creates (I think, from other excellent posts in here) an 'apples to oranges' comparison with my other regressions that used the intercept. In one case I removed it and $R^2$ improved greatly and in another it did not. I'd like opinions on:

  1. Should I not use $R^2$ and use AIC for model evaluation across the board so I can generate/evaluate models utilizing only statistically significant intercepts and slopes?
  2. Are there other methods/metrics for model evaluation for differing explanatory variables given the same response variable in a linear model?
  3. I can use the glm() in R which would give the AIC, yes?
  4. Looking at the math, the RSE should still be a valid consideration in either the intercept or non intercept case yes?
  5. Does anyone have some self study sources on the topic? I'd like to feel less like a monkey with a typewriter using this wonderful program and not generate statistical abuse in the process.

Thanks guys. As you can tell sort of novice here and your opinions are well received.

Sorry, can't post code or data in here - confidentiality agreement.

$\endgroup$
2
  • $\begingroup$ If we view (5) as the question, this might be on topic. Otherwise it is too unfocused and would be closed. Please see our help center for more about this. $\endgroup$
    – whuber
    Oct 29, 2021 at 15:31
  • $\begingroup$ How does the intercept not being significant (…and at what $\alpha?$) influence how you would compare modes? // When you remove the intercept in R, the $R^2$ calculation in “summary” is different and should not be compared to an $R^2$ value coming from a model with an intercept. $\endgroup$
    – Dave
    Oct 29, 2021 at 15:39

1 Answer 1

0
$\begingroup$

Think about what $R^2$ does.

$$ R^2 = 1- \dfrac{ \sum (y_i-\hat y_i)^2 } { \sum (y_i - \bar y)^2 } $$

$R^2$ compares your performance to the performance of a model that always predicts the mean of $Y$.

This makes sense to me. If you are trying to predict the conditional mean, the most reasonable naïve way of guessing the conditional mean of $Y$ is to use the marginal/pooled mean of $Y$.

When you remove the intercept from the model, you’re implying that the marginal mean of $Y$ is zero, and R accounts for this by tweaking the $R^2$ calculation.

$$ R^2_{tweaked} =1- \dfrac { \sum (y_i -\hat y_i)^2 }{ \sum (y_i-0)^2 } $$

When you say that removing the intercept improves the $R^2$ of your model, you are comparing two different calculations, almost as much as if you evaluated one model on RMSE and another on MAE; they’re simply different measures of performance. If you do the calculations yourself (or maybe R has a way to force it not to tweak the $R^2$ calculation), you will find in all of your models that the usual $R^2$ calculation gives you better performance when you include the intercept.

For the most part, unless there is a strong theoretical reason for fixing the intercept at zero, most suggest to leave it in the model. After all, you know the intercept probably isn’t exactly zero, which is what excluding the intercept says.

$\endgroup$
2
  • 1
    $\begingroup$ Why did this post as a community wiki? Is it because the OP is community wiki? Mods, can this be undone? $\endgroup$
    – Dave
    Oct 29, 2021 at 15:53
  • 1
    $\begingroup$ Thanks Dave. Really appreciate the answer regardless of my late response. I looked a bit at what you illustrated above and did some of the math/proofs and it makes perfect sense. Thanks for the direction. It's funny, when you start doing something you don't know what you don't know. Appreciate the illumination. $\endgroup$
    – S. Pitcher
    Nov 18, 2021 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.