Minimum number of observations for multiple linear regression

Question

I am doing multiple linear regression. I have 21 observations and 5 variables. My aim is just finding the relation between variables

1. Is my data set enough to do multiple regression?

2. The t-test result revealed 3 of my variables are not significant. Do I need to do my regression again with the significant variables (or my first regression is enough to get conclusion)? My correlation matrix is as follow

          var 1   var 2    var 3   var 4   var 5     Y
   var 1   1.0     0.0       0.0   -0.1    -0.3    -0.2
   var 2   0.0     1.0       0.4    0.3    -0.4    -0.4
   var 3   0.0     0.4       1.0    0.7    -0.7    -0.6
   var 4  -0.1     0.3       0.7    1.0    -0.7    -0.9
   var 5  -0.3    -0.4      -0.7   -0.7    1.0      0.8
   Y      -0.2    -0.4      -0.6   -0.9    0.8      1.0
   

var 1 and var 2 are continues variables and var 3 to 5are categorical variables and y is my dependent variable .

It should be mentioned the important variable which has been considered in the literature as the most influential factor on my dependent variable is not also among my regression variables due to my data limitation. Does still make sense to do regression without this important variable?

here is my confidence interval

    Varibales   Regression Coefficient  Lower 95% C.L.  Upper 95% C.L.
    Intercept   53.61                       38.46        68.76
    var 1       -0.39                      -0.97         0.19
    var 2       -0.01                      -0.03         0.01
    var 3        5.28                      -2.28         12.84
    var 4       -27.65                     -37.04       -18.26
    **var 5      11.52                      0.90         22.15**

Answer 1

The general rule of thumb (based on stuff in Frank Harrell's book, Regression Modeling Strategies) is that if you expect to be able to detect reasonable-size effects with reasonable power, you need 10-20 observations per parameter (covariate) estimated. Harrell discusses a lot of options for "dimension reduction" (getting your number of covariates down to a more reasonable size), such as PCA, but the most important thing is that in order to have any confidence in the results dimension reduction must be done without looking at the response variable. Doing the regression again with just the significant variables, as you suggest above, is in almost every case a bad idea.

However, since you're stuck with a data set and a set of covariates you're interested in, I don't think that running the multiple regression this way is inherently wrong. I think the best thing would be to accept the results as they are, from the full model (don't forget to look at the point estimates and confidence intervals to see whether the significant effects are estimated to be "large" in some real-world sense, and whether the non-significant effects are actually estimated to be smaller than the significant effects or not).

As to whether it makes any sense to do an analysis without the predictor that your field considers important: I don't know. It depends what kind of inferences you want to make based on the model. In the narrow sense, the regression model is still well-defined ("what are the marginal effects of these predictors on this response?"), but someone in your field might quite rightly say that the analysis just doesn't make sense. It would help a little bit if you knew that the predictors you have are uncorrelated from the well-known predictor (whatever it is), or that well-known predictor is constant or nearly constant for your data: then at least you could say that something other than the well-known predictor does have an effect on the response.

Answer 2

The answer to the general question is that it depends of many factors with the main ones being (1) number of covariates (2) variance of the estimates and residuals. With a small sample you do not have much power to detect a difference from 0. So I would look at the estimated variance of the regression parameters. From my experience with regression 21 observations with 5 variables is not enough data to rule out variables. So I would not be so quick to throw out variables nor get too enamored with the ones that appear significant. The best answer is to wait until you have a lot more data. Sometimes that is easy to say but difficult to do. I would look at stepwise regression, forward and backward regression just to see what variables get selected. If the covariates are highly correlated this may show very different sets of variables being selected. Bootstrap the model selection procedure as that will be revealing as to the sensitivity of variable selection to changes in the data. You should calculate the correlation matrix for covariates. Maybe Frank Harrell will chime in on this. He is a real expert on variable selection. I think he would at least agree with me that you should not pick a final model based solely on these 21 data points.