1
$\begingroup$

I have some survey data collected from 15 government organizations. Using this data, I would like to run an OLS regression to estimate the influence of variables x1 and x2 on y. I have run two models: one including 14 dummy variables accounting for potential differences from organizational membership, and one not including any dummy variables. The difference between R-square values between the models is small (less than 0.02), and the adjusted R-square difference much smaller. Moreover, the coefficients for x1 and x2 and significance levels are not substantially different between models, and none of the dummies has a statistically significant relationship with the dependent variable.

I am hoping for some insight into the following questions:

  1. From a theoretical standpoint, is it necessary to include dummy variables for organizational membership (in order to protect against omitted variable bias or some other reason)? In other words, must I include the dummy variables on principle?

  2. If I must include them on principle, is there any post analysis test that I can use to claim that the model without the organizational dummies should be used in the interpretation of the results? (Something similar to the Hausman test for distinguishing between fixed and random effects models, for instance.)

Please let me know if I can provide any more information that will help.

Improve this question
$\endgroup$
2
  • $\begingroup$ maybe you tried already, but if your independent vars are the same for every organization maybe you could use a nested model or a semplification. Nesting with random effect takes in consideration that the variance is different into the clusters (say, math votes of kids in class are more similar than kids in another class). Alternative would be averaging your data inside clusters. But this way you reduce your observations and loose significancy (2° type error). The opposite (not considering any clustering) boost observation and therefore significancy (1° type error) $\endgroup$
    – Bakaburg
    Commented Dec 18, 2014 at 13:47
  • $\begingroup$ So a nested model is good in between solution $\endgroup$
    – Bakaburg
    Commented Dec 18, 2014 at 13:47

1 Answer 1

Reset to default
4
$\begingroup$

I do not see any statistical need to include such dummy variables, given what you've said.

However, if others doing research in your field would regard them as substantively necessary, it still might be a good idea to include them. In addition, if others have found large effects where you find small ones, that may be interesting as well.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thank you for this answer. I suppose that there is theoretical reason enough to include the organizational controls. As to the second part of the question, perhaps an insignificant incremental F test would demonstrate the lack of explanatory power in the set of indicator variables? I will follow that path for the time being. $\endgroup$
    – kwela12
    Commented Sep 9, 2013 at 16:14
  • $\begingroup$ I would not use the F test to demonstrate this, I would use the parameter estimates for the dummy variables, and the lack of change in the other parameter estimates on their inclusion or exclusion. $\endgroup$
    – Peter Flom
    Commented Sep 9, 2013 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.