3
$\begingroup$

When doing imputation, I can understand why mean substitution can result in regression dilution.

However, in the same article about imputation, I don't understand why this is not a concern for univariate analysis?

As more of the y-values (dependent variables) get replaced with their means, I know that the sample mean will stay the same (and hence the bias will not be affected). But, it seems to me like this will increase the variance (and standard deviation), regardless of if it is univariate or multivariate. Am I not understanding this correctly?

So, when the wikipedia article says that "mean imputation has some attractive properties for univariate analysis but becomes problematic for multivariate analysis", I don't think that's true. seems to me like they have exactly the same problem?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ Not sure what the context is of your linked articles, but typically regression of a single response variable on a single predictor variable would be considered bivariate, not univariate. (In univariate analysis mean-substitution would result in attenuation of e.g. standard error estimates, so inflation of t-statistics.) $\endgroup$ – GeoMatt22 Dec 12 '16 at 2:49
  • $\begingroup$ Thank you @GeoMatt22. I fixed the question now. It's really that I don't understand why both univariate and multivariate analyses would have the same issue with mean imputation. $\endgroup$ – Sother Dec 17 '16 at 21:14
2
$\begingroup$

I think the wiki author is somewhat limited in their knowledge of mean imputation.

If I were fitting a regression model for the mean of $Y$ as a linear combination of $X$ and $W$, but $X$ has missing values, mean imputation would not impute the missing values of $X$ with $\bar{X}$. I would have to impute those values with $E[X|W, Y]$ to get an unbiased estimator. In bivariate analyses, imputing $\bar{X}$ has no impact on the slope $\hat{\beta}$ because it is a 0 leverage point. Not true in higher dimensions, but still the wrong approach regardless.

Mean imputation is generally bad practice. Using a proper conditional mean imputation, while unbiased, tends to underestimate the standard errors. Mean imputation fails to take into account any possible sampling variability in the missing values. Multiple imputation provides far better estimation and is not biased.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ When I use the word "bias", I mean the formal definition from statistics. It seems to me like mean imputation will not bias the mean. The sample mean is an unbiased estimator of the population mean. $\endgroup$ – Sother Dec 17 '16 at 21:26
  • $\begingroup$ Okay. When you condition the missing values on the covariates which are present and the label, $E[X | W,Y]$ what you're describing is actually called regression imputation not mean imputation. Getting confused between the difference between those now, unless wikipedia is wrong. $\endgroup$ – Sother Dec 18 '16 at 23:02
  • $\begingroup$ @Sother not necessarily but usually yes. You need a conditional mean, which is what $E[X|Y,W]$ would be here. You can estimate that from regression in the same dataset, but you can also use whatever prediction model you want (Bayesian, machine learning, etc.) the most important thing for mean imputation is that the conditional mean is a good predictor. I don't really know what Wiki is saying, but if you fill in the missing values of $Y$ with $E[Y]$ and not $E[Y|X]$ I can see how that would cause "dillution". $\endgroup$ – AdamO Dec 19 '16 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.