Regression Hypothesis Testing for multivariate models

Question

I am running some linear regression models in R using the lm function and calling the summary() function on my linear model gives a nice summary of the degrees of freedom, F-statistic, sum of squares, etc.

Call:
lm(formula = y ~ a + b + c + d, data = my_data)

Residuals:
       Min         1Q     Median         3Q        Max 
-0.0313491 -0.0034180 -0.0000105  0.0030041  0.0306064 

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)     1.004e-04  3.396e-04   0.362    0.717    
a              -2.739e-02  0.893e-02  -1.447    0.149    
b               5.019      5.127e-01  19.249   <2e-16 ***
c               7.090      2.954e-01  31.171   <2e-16 ***
d               4.241      2.760e-01  22.978   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.006236 on 354 degrees of freedom
  (1 observation deleted due to missingness)
Multiple R-squared:  0.881, Adjusted R-squared:  0.8797

Now, this gives p-values / significance tests based on the null hypothesis that: $$H_0: \mu_1 = \mu_2 =\mu_3 =\mu_4 = 0$$

Therefore, since each p-value is statistically significant at $\alpha=0.05$, we can reject the null hypothesis and conclude there is a linear relationship between each variable and the response variable.

Now I am interested in determining if one variable is statistically different from another variable, i.e. that it my model can be simply explained by just $b$ or just $b$ and $c$, etc. (I guess this is the same as determining if a two variables are co-linear but I am uncertain.)

What tools can I use clarify this and find the most significant model?

Answer 1

You are looking at comparing models, not comparing covariates really. If you have the models in mind, that is, you want to compare model $a$ to model $b$, you can use AIC, which estimates the fit of a model while taking its complexity into account: aic(model_a,model_b) and choose the lower outcome AIC model.

Another method, is using the adjusted $R^2$ of all the models you want and compare them. Generally, the model with the smallest amount of covariates and the highest adjusted $R^2$ is your winner. If you want to check all covariate combinations instead of manually choosing specific models to compare, you can use an all subsets regression from library(leaps) in R. This will allow you to choose the number of best combinations to show, which you can also plot() for a visual aid. If this is what you need, see this great tutorial from R-studio pubs.

It might be worth noting that choosing the fittest model should be done with care. I come from social sciences, and the way I see it, sometimes it is worth adding non-significant covariates to the model - it they have a meaning. As a simple example, if I check for predictors of salary, and gender is non-significant, than perhaps some other covariate made it non-significant. If so - that is of interest. If not, than perhaps there is no gender pay-gap - also interesting. My point is Choose a model with care, not just by looking at the numbers - they do not always convey the entirety of the picture.