Is choosing only significant coefficients (based on t stats) from a multiple linear regression model a good idea, provided the F stats is significant?

Question

I doubt if this topic has already been discussed here. I did search the forum before posting this question and read similar posts, however unable to find my answer, perhaps due to my limited understanding (most of them were discussed around multicollinearity).

I have received the following dataset from our economics Professor. It has 15 observations and 4 variables - qsold (quantity sold of product X), psn (price of X), pcb (price of a substitute product Y), adv (expenditure on advertising of X). I am supposed to derive a demand function (qsold = B0 + B1 (psn) + B2 (pcb) + B3 (adv)). Now theoretically, all three independent variables are supposed to have a relationship with qsold, however, I am supposed to explore only linear relationship, so, I tried to fit the following model.

df1
qsold   psn pcb adv
1183    1361.97 1405.78 3.22
974 1520.49 1369.17 3.39
1179    1361.43 1448.71 4.03
1258    1159.67 1465.12 3.91
1161    1297.74 1383.93 3.46
1052    1362.44 1450    3.64
992 1447.25 1404.4  3.55
1213    1316.93 1418.03 3.81
1133    1365.97 1391.95 4.21
1001    1283.92 1403.11 4.22
1221    1329.34 1428.9  3.38
1137    1278.41 1426.81 3.89
1112    1466.21 1442.68 3.65
1025    1355.73 1359.79 4.25
1277    1377.06 1455.03 3.35

Call:
lm(formula = qsold ~ psn + pcb + adv, data = df1)

Residuals:
    Min      1Q  Median      3Q     Max 
-118.47  -31.59   12.42   39.46   92.43 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)  
(Intercept)  635.4451  1240.7873   0.512   0.6187  
psn           -0.5897     0.2647  -2.228   0.0477 *
pcb            1.1835     0.6650   1.780   0.1027  
adv         -103.7231    62.7722  -1.652   0.1267  
---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 72.57 on 11 degrees of freedom
Multiple R-squared:  0.5764,    Adjusted R-squared:  0.4609 
F-statistic:  4.99 on 3 and 11 DF,  p-value: 0.02004

In the output above, only psn is significant (based on t stats). Our Professor told us that we should consider only significant coefficients in the demand function. I am finding it difficult to agree with. In this case, if I consider only psn in my demand function, I am perhaps violating the null hypothesis, which is based on F stats (that all coefficients are 0). Also, if I consider only psn I am essentially negating the combined effect of all three variables, and basically choosing a different model that what I fitted. Please provide your inputs if you think this question is worth discussing. Also, if possible please cite literature, which I can refer.

Answer 1

Although this may have been answered elsewhere: No, it's not a good idea. Reasons:

1. The standard t-test tests whether a nonzero coefficient is needed assuming that all the other variables are in the model. It may be that two or more variables share the same information about the response. These may all not be significant in the full model because the information is also present in the respective other variable(s). However, if you remove them all at the same time, you lose that information. (This motivated schemes such as backward elimination eliminating variables one by one, although these don't solve all problems either and are rather outdated these days; but they're surely better than removing all insignificant variables at once.)

2. There is no general reason to remove insignificant variables from the model at all. Removing a variable means that you set its coefficient to zero. But the original estimator in the full model is the best guess of what this coefficient is, so it can be expected to be better than "precisely zero" when it comes to prediction, and also when it comes to adjusting for the influence of that variable when estimating the (supposedly) more important ones, significant or not. In my view (which agrees with the Frank Harrell book recommended in the comments) there needs to be a positive reason to remove variables such as (a) there are so many variables in the model that the regression estimate is too unstable or cannot even be computed due to collinearity, or (b) measuring the variables in the future would be expensive and one would prefer a model with fewer variables for that reason, or (c) there are good subject matter reasons to believe that some variables are just noise, or (d) one can show (by means of cross-validation and the like) that prediction quality improves without certain variables. Some also think that it would be better to have a smaller model for easier interpretation, but one can fit the full model and put the focus of interpretation on the significant variables with insignificant ones still there. There's no need to remove variables for that (maybe this is what your professor wants you to do?).

3. Generally it is not a statistical indicator of quality of a model that it has only significant variables. The tests give you information about the strength of evidence about the parameter values, and both significant and insignificant tests are informative and OK to have.