Let's just say that in my field several studies find some significant associations between a variety of CEO attributes and some organizational policy (binary categorical dependent variable). Many use logistic regression and then infer that any statistically significant attribute is a causal factor for why CEOs are more likely to adopt certain policies.
What I have found however is that there may be reason to question whether the policies are even a product of CEO decision-making. A through review of the cases reveals that in ~30% of them, the policies are actually initiated by middle-management and not the CEOs.
So my hypothesis is that by including all cases, as previous research does, results in biased estimates. I predict that when re-running their analyses with only relevant positive cases where CEO decision-making mattered, we should then see different effects or even non-significant associations with the various CEO attributes. I am struggling, however, with three things:
So how can I better explain the problem in more formal methodological terms? I know measurement error on the DV does not bias estimates for the IVs in interval data, but this doesn't really make sense in my case since the DV is categorical. Is this misclassification? measurement error, Type 1 error, or something different?
What is the best way to test this hypothesis using statistical methods?
In brief, my strategy is to run two (logistic) models, one where all positive cases are included and another where only relevant positive cases are included, and then compare the models based on change in effect sizes, significance levels, and McFaddens's pseudo R-squared. Only problem is I know the significance level will decrease regardless because there will be less positive observations.