I am conducting multivariate regression analysis. Say my model is (i index suppressed for simplicity):

$$Y = \beta_{0} + \beta_{1}X_{1} + \beta_{2}X_{2} + e$$

I have a binary variable $Z$ which indicates (something like) the quality of observations. In principle, I want to eliminate those with low quality ($Z=0$) and run my model only with high quality observations. Yet, I want to provide formal evidence that this is a good move. Particularly, I want to test whether quality is not correlated with $Y$, $X_{1}$, and $X_{2}$. This would imply that quality is randomly distributed across the sample, and thus dropping low quality observations will not bias the estimation.

(Although in principle you might consider randomness as a reason not to drop them, the problem with low quality data is high variance, perhaps due to random measurement errors).

My approach so far has been to conduct simple t-tests (equivalent, afaik, to one-way ANOVA) comparing means of $Y$, $X_{1}$ and $X_{2}$ among both quality groups.

My question are: can I do this in a joint fashion? Testing all three together? Is this better or is my approach enough?

The reason I am thinking on this joint test is because, in RCT studies, it is common to regress the instrument on observable characteristics, to check that it is actually randomly distributed. I'm trying to see a parallel with that approach here.


One possibility I can think of would be to run a logit (or probit) regression with $Z$ as dependent variable and $Y$, $X_1$ and $X_2$ as independent variables (and combinations of these if it makes any sense). This way you check whether you can predict which observations have a low quality based on their characteristics / outcome.

  • $\begingroup$ So basically, perform the same test that is conducted in RCT studies? $\endgroup$ – luchonacho May 19 '16 at 13:44
  • $\begingroup$ Yes... and by the way if there is no correlation and you don't drop too many points... your estimation results should not vary too much. Otherwise, as a reader, I'd expect to find both formal evidence and solid qualitative arguments to support data restrictions. $\endgroup$ – etna May 19 '16 at 17:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.