3
$\begingroup$

I have performed a regression with the explanatory variables by themselves and they do not turn out to be significant. When control variables are included, the control variables themselves turn out to be significant but the main explanatory variables remain insignificant. How does one interpret such a result? Does this mean the model is misspecified and that the variables of interest are the control variables?

$\endgroup$

3 Answers 3

4
$\begingroup$

One interprets these results the same way as any other multiple regression. Of course, it might be that your model is misspecified, but calling a variable "main" or "control" is not a misspecification of a model. The regression doesn't "know" which are main and which are control variables. But you might need more control variables, or you might need a bigger sample, or you might have actually misspecified things and the relationship you are looking for is non-linear .... Or, just possibly, your theory is wrong and your explanatory variables do not explain things very well.

$\endgroup$
1
$\begingroup$

In the context of regression control variables play a specific role, usually with respect to causal inference goal. Control variables are needed if without we have reasons to suspect that the coefficients of target variables are biased. So:

... Does this mean ... that the variables of interest are the control variables?

No. What are the variables of interest, or better the effects of interest, should be decided before to perform any regression. Moreover even the choice of right control set of variables should to be discussed before to perform any regression, at least ideally.

The result you describe, not significant estimated target coefficients, should be interpreted that the effects you looking for are simply absent. If you are convinced otherwise, rethink control set can be an idea. Maybe the entire specification can be rethought.

$\endgroup$
1
$\begingroup$

The statistical significance of predictors can be affected by many factors. Some factors related to the inclusion of covariates/controls include but are not limited to:

  • Magnitude of effects. As the magnitude of the effects from a variable increases, this increases the chances of getting statistical significance. If the relative magnitude of effects is large in one predictor and weak in another, this will also determine the statistical significance of some predictors over others.
  • Sample size / observations. As the sample size or number of observations increase, the coefficients become more likely to be statistically significant purely via reduction of the standard error in each coefficient. With the addition of more control variables, smaller samples can also contribute to overfitting if the observations per parameter are very low, which may also affect the $p$ values in the model.
  • Multicollinearity. If two more more variables in a regression are perfectly or almost perfectly correlated, this can create inflated standard errors and with it a lower prevalence of statistically significant results. For example, suppression effects can make a statistically significant predictor non-significant after the inclusion of another highly correlated variable.
  • Omitted variable bias. The omission of key predictors can of course influence the statistical significance (and interpretation) of the regression as well. Misspecification of the model in general can affect significance in different ways.

How this affects your regression and which variables enter it will vary case by case, but remember that a regression simply reconstructs $y$ by splitting it up by it's conditional mean (the intercept) and whatever variables weight this relationship (the slopes). As such, the differences in predictors will vary a lot and contribute to changes in statistical significance. One can look at the list above and test how the inclusion of more predictors may be weighted by some of these factors.

It is important for me to emphasize that statistical significance does not mean practical significance (see here and here for more on that), so one should not freak out if control variables alter the results. Simply report them as they are, visualize these effects in whatever way you can, and determine a theoretically justified explanation of what you see. Often a picture is worth a thousand words, so plotting these relationships will help elucidate what effects are present. As an example, plotting the standard error around the regression line will tell you more than any $p$-value will.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.