How to chose covariates to adjust for in a regression analysis

Question

I am running a regression analysis where my primary interest is to see if the outcome differs by the group (treatment vs. control). However, I have some 80 other clinical and socio-demographic variable that could potentially contribute to the outcome (no literature or theory, selection of these covariates are exploratory). I do not want to include all of these predictors in the regression model, rather would like to screen which ones to include based on some criterion.

Is looking at the bivariate correlation between the outcome and covariates is a good strategy to select potential predictors?

Also, for interaction, can I look into the association between the group and other covariates, and include the interactions where the association is significant?

Open to suggestions. Thank you in advance!

Answer 1

Firstly you could compute a correlation matrix with the Y (dependent) in the first column and all the Xs (predictors). Look along the first row or column and eliminate the Xs that have low pairwise correlation with Y. So that may give you an initial reduction in the number of predictor variables. Regardless of whether you do this first step or not, you should: firstly do backward step-wise regressions, followed by forward step-wise regressions. I'll assume that you want a final model with 30 predictors. So the recipe goes like this: Subtract the mean of each variable (including Y) from the raw data, so that until you settle on a final model you needn't concern yourself about an intercept term.

1) Regress Y an all 80 Xs. Compute the t-stats for all 80 coefficients. Make note of the variable with the lowest t-stat and the R-square achieved. Then regress Y on the Xs again but this time exclude the predictor with the lowest t-stat from the previous regression. Again make note of of the variable with the lowest t-stat and the R-square achieved. The third regression will be run with 78 predictors having again eliminated the variable with the lowest t-stat from the previous regression. Repeat until you have a regression with only one predictor.

2) Now work in the opposite direction. Regress Y on one X variable, trialing all 80 predictors and select the X that gives you the highest R-square. Again keep track of your variables included and excluded and the R-square. The the second regression will include the first X selected and all of the remaining 79 are trialed to find the two predictor regression with the highest R-square. Compare this to the two predictor regression R-square achieved in 1) above. If you achieved a higher R-square in the backward step-wise regression then this is selected as the best variable selection moving forward. Adjust your included/excluded predictor schedule accordingly. Then the third regression will be the regression that gives you the highest R-square from the 78 Xs not included so far, and again compare to results given in 1). Repeat until you have a regression with your target 30 predictors.

The recipe given above almost invariably gives you the best variable selection. 1) and 2) may give similar selections but often the forward step-wise regressions in step 2) will provide some improvement.