5
$\begingroup$

In my data, some individuals have missing data on the central predictor (father missed the intake assessment). Comparing the DVs' means for those with a missing/non-missing predictor yielded some sizeable effects.

Now I want to find out whether the systematic missings may have led me to underestimate the size of the OLS regression coefficients. What's a good way to do this?

Simply comparing the variances of the DVs in the group without missings to the group with missings is easy to do?
But conceptually I want to know whether the whole sample has significantly less variability when I leave the group with missings (and significantly-lower-than-average-scores) out, not whether the two groups (with missings and without missings) have different variances.
All that I found so far was about independent samples, not about subsets.

Also, just to bring me up to speed: heteroscedasticity is usually used in the context of residual variance, right? What's a good term for constricted variance that would give me better luck with google?

Improve this question
$\endgroup$
5
  • $\begingroup$ Just to help understand the problem - why do you care about range restrictions but not so much about variance? And why do you think there's any restriction on the range at all, as contrasted with simply less variability / fewer observations leading to a slightly tighter range of the DV? $\endgroup$
    – jbowman
    Commented Jul 24, 2012 at 2:00
  • $\begingroup$ @jbow I don't care more about range than variance. I want to know whether I may have underestimated the estimated OLS relationships because I had to exclude this group of data points with significantly-lower-than-average-scores. I'm quite certainly using the wrong terms, judging by what turns up when I google. I'll rephrase and gladly use correct terms (I first tended towards "constricted variance" but that led to only 85 hits). $\endgroup$
    – Ruben
    Commented Jul 24, 2012 at 8:18
  • $\begingroup$ @jbow That is, if someone can give me any pointers what the correct terms would be. $\endgroup$
    – Ruben
    Commented Jul 24, 2012 at 8:27
  • $\begingroup$ To summarize: your concern is that a) the missing data does not appear to be missing completely at random (acronym: MCAR) and therefore b) your parameter estimates may be biased? If so, that's certainly a valid concern! $\endgroup$
    – jbowman
    Commented Jul 24, 2012 at 14:19
  • $\begingroup$ @jbow Yes, and I'd especially like to know whether I may have underestimated my parameters (biased towards towards the null) because of limited variance (because I lost many values in the lower end of the distribution). $\endgroup$
    – Ruben
    Commented Jul 24, 2012 at 22:49

1 Answer 1

Reset to default
1
+50
$\begingroup$

I think you have posed two different questions: one which asks about bias in your coefficients and another which talks about variance.

Bias

As far as I can tell, the general solution to attempting to understand the likely bias in coefficient estimates is to explicitly model the cause of the missing data. However, in most situations that I have come across such information is not available and there is instead a need to do something a little more pragmatic (but imperfect). The following should tell you if you have bias:

  1. Recode the missing data on the central predictor by assigning the mean to each observation.
  2. Create a new binary variable which has a 1 when respondents are missing data and a 0 otherwise.
  3. Re-estimate your models with these two independent variables replacing the central predictor.

The coefficient of the binary variable will capture the phenomena of missing group having a different mean on the dependent variables. You can assess bias by comparing the coefficients obtained with the models I have just described with the models you were obtaining with the smaller sample size.

Variance

A priori, adding in the respondents with the missing data is going to increase your variance. If they were the same as everybody else then your variance wouldn't change, but you know they are not the same as everybody else so your variance should increase. However, I doubt this is so relevant to solving your problem as your interest is the conditional variance (i.e., the relationship between the predictors and your dependent variables) rather than the variance per se.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Thanks for clearing that up. You're correct in saying that the conditional variance matters more, I just read this remark "restricted variance may have caused us to underestimate the relationship" too often. Actually, I often have data on the central predictor from another source, more often a vital covariate is missing. Would I follow the same procedure? I can actually in some cases model the reason for missing data (ie. dad dead, unknown, didn't show). $\endgroup$
    – Ruben
    Commented Jul 25, 2012 at 11:58
  • $\begingroup$ You helped me a lot, especially with the part about conditional variance, thanks. I thought about it some more, and read up on FIML and remembered all that MAR/MCAR jazz. I may have a follow-up question regarding NMAR vs. MAR as a FIML assumption, but that's for another post. $\endgroup$
    – Ruben
    Commented Jul 26, 2012 at 21:20
  • $\begingroup$ I'm not entirely sure what you mean by "I often have data on the central predictor from another source". However, as regards the missing data assumptions, in problems that I model I generally always assume that the data is MAR as I tend not to have sufficient data to do anything else, so it sounds like you have data that will permit you to do something outside of my experience. I'd suggest you have a flick through Little, Roderick J. A. and Donald B. Rubin (1987), Statistical Analysis with Missing Data. Brisbane: John Wiley & Sons. It is not so new, but I always find it really useful. $\endgroup$
    – Tim
    Commented Jul 27, 2012 at 3:52
  • $\begingroup$ Thanks for the reference, can't find a PDF, but I'll look it up in the lib. I meant that: Initially I did complete-cases analysis and because I definitely had information on a vital covariate only from that assessment, I didn't bother to look for the other predictors in the other assessments. I did that now and now have 440 fewer (dataset of ~1800) missings on the central predictor, but still ~600 missings on the covariate. I can predict up to 38% of the variance in that oft-missing covariate using the other variables in the model and FIML seems a good approach, even when NMAR... Reading up.. $\endgroup$
    – Ruben
    Commented Jul 27, 2012 at 9:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.