3
$\begingroup$

Say I have an OLS regression with a dummy variable level A, B, C and D, where A is the reference category. Will the estimated coefficient value and/or statistical significance of B or C change or be impacted if I remove from the input data set all of the data mapped to D and re-run the regression (which will no longer contain dummy variable level D)?

On the values changing or not, I would think not because the coefficients of B and C simply reflect the difference between the mean of B and C, respectively, and the mean of A (the reference).

Is my understanding correct?

$\endgroup$
4
  • $\begingroup$ this question may help quora.com/… $\endgroup$
    – aghd
    Sep 23, 2019 at 5:03
  • 3
    $\begingroup$ I think you need to clarify what you mean by 'dropping' to judge whether Noah's answer or mine is correct. I interpret you to mean that you are fitting the model after removing all data labelled D. $\endgroup$
    – mkt
    Sep 23, 2019 at 5:17
  • 1
    $\begingroup$ @mkt makes a good point that I had not thought of. This could mean two different things. $\endgroup$
    – Peter Flom
    Sep 23, 2019 at 11:54
  • 1
    $\begingroup$ Good point. I reworded the question. @mkt interpretation of the original question is correct. $\endgroup$ Sep 23, 2019 at 15:05

2 Answers 2

6
$\begingroup$

I interpret what you are doing as removing all data associated with the label 'D' from the dataset before fitting the model. This is distinct from relabelling 'D' to 'A' or some equivalent step, which is Noah's assumption.


It depends on whether you have any other variables in the regression. If you do not, then your interpretation is correct. Dropping D will not affect the intercept (A), or the coefficients for B or C.

But imagine you also have an additional variable that you are using as a predictor. If the distribution of this continuous variable is non-random with respect to D vs. A, B, or C (i.e. they are associated in some way), then dropping D may also change the coefficient for the additional variable. This change may also affect the coefficient estimates and associated p-values for A, B, or C if they are associated with the additional variable.

$\endgroup$
14
  • $\begingroup$ Thanks. You mention "It depends on whether you have any other variables in the regression", you seem to imply this would only matter if the 'other variables' are continuous. But what if I have another dummy set, it's not clear to me why the levels of this second dummy set would be impacted. $\endgroup$ Sep 23, 2019 at 5:01
  • $\begingroup$ @StatsScared Whether it is continuous or categorical does not matter. I was using continuous for ease of illustration, but have removed that detail now. $\endgroup$
    – mkt
    Sep 23, 2019 at 5:14
  • $\begingroup$ @mkt, thanks for the response. However, I am still unclear on whether the coefficients for B or C will be impacted - both values and statistical significant - if D is dropped and the regression includes a continuous variable and/or another dummy set (one that is different from A, B, C, D). $\endgroup$ Sep 23, 2019 at 15:57
  • $\begingroup$ @StatsScared Coefficients and p-values may well be affected in the situation you describe, but one cannot say exactly in what way. That's because it depends on the strength and nature of the association between the predictor variables. I've edited my answer to make that more clear. $\endgroup$
    – mkt
    Sep 23, 2019 at 15:59
  • $\begingroup$ @mkt Thanks. I guess a side question is how would two sets of dummies be related to one another so that there is some association between them? I thought this 'association' would only take place if there was an interaction between levels of each dummy variable set. $\endgroup$ Sep 23, 2019 at 17:45
0
$\begingroup$

Edit: OP clarified their question, so this answer does not apply. This answer responds to the question of simply removing the dummy variable for category D from a regression that contained the full dataset.


Dropping categorical D is equivalent to relabeling all D's as A's. This will absolutely affect your intercept and coefficients for the non-reference categories. The intercept will now be the mean of those in categories A and D combined, and the coefficients on the non-reference categories will be the difference in means between the corresponding category and the combined A and D group. The only reason it would make sense to do this would be if you wanted to assume that the means for categories A and D were equal. If they were equal in the population, you would have some increased precision in your estimates. If not, you will have biased estimates for the true coefficients.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.