I have created 2 linear regression models on a data set and its extract (I removed some features from teh first data set for this second model as the number of samples was too small). None of them gives significant coefficients: Model1:
Residuals:
Min 1Q Median 3Q Max
-42.812 -25.919 -3.612 12.394 70.756
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 66.9098816 44.4508422 1.505 0.150
x1 0.0332949 0.4990344 0.067 0.948
x2 -4.1744552 6.7686418 -0.617 0.545
x3 -2.0553138 11.1224723 -0.185 0.855
x4 -4.0994890 15.9008763 -0.258 0.799
x5 -0.0068639 0.0798893 -0.086 0.932
x6 -0.1989496 2.2244522 -0.089 0.930
x7 -0.0103695 0.0459473 -0.226 0.824
x8 0.0009110 0.0010026 0.909 0.376
x9 -0.0001616 0.0006995 -0.231 0.820
Residual standard error: 38.12 on 18 degrees of freedom
Multiple R-squared: 0.1323, Adjusted R-squared: -0.3015
F-statistic: 0.305 on 9 and 18 DF, p-value: 0.9633
And for Model2 I have:
Residuals:
Min 1Q Median 3Q Max
-47.48 -26.48 -7.77 16.99 75.94
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 77.73117 36.14409 2.151 0.0428 *
x1 0.04839 0.37977 0.127 0.8998
x2 -7.12185 5.57618 -1.277 0.2148
x3 -2.23965 12.26234 -0.183 0.8567
x4 -0.03269 0.06258 -0.522 0.6066
x5 -0.01020 0.03985 -0.256 0.8004
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 35.42 on 22 degrees of freedom
Multiple R-squared: 0.08402, Adjusted R-squared: -0.1242
F-statistic: 0.4036 on 5 and 22 DF, p-value: 0.8411
I have 28 samples in my data set and I know it's better to use cross-validation. But I was wondering which of these models are better model in general? Can I rely on a model if the RMSE is small compared to the range (or mean) of the output variable but none of the coefficients of the variables is starred (significant)?