I have been told that mean imputation of missing values is inappropriate when the variables underlying distribution is non-normal. my variable is contiunous (but bound at 100) and most observations are either 98 or 99 (with the odd few in the lower 90's), hence the distribution is highly skewed. how would i best input for missing values?
2 Answers
The typical modern approach in this sort of situation is to use some form of multiple imputation.
The general idea is that instead of imputing just one 'best' value to the missing data, you repeat an imputation process many times using the known statistics of the missing value (including considering whether the missing data might be correlated with other variables that are not missing), generating multiple distinct sets of data. Then you run your analyses on each imputed copy of the data, and finally pool those analyses together.
MICE is a popular implementation in R.
You can expect that other major statistical packages like Stata will also have functions for multiple imputation.
-
$\begingroup$ That's brilliant, thank you! is there an equivalent command in STATA? $\endgroup$ Commented Jul 24, 2018 at 22:01
-
1$\begingroup$ @TomWitten See edit. I'm not as familiar with Stata but there seems to be plenty of support to get you started. $\endgroup$ Commented Jul 24, 2018 at 22:04
I suggest that you implement multiple imputation, specifically the 'Predictive Mean Matching' algorithm. A highly relevant reference to your problem is given below, which provides empirical evidence of this method performing reasonably with various levels of skew in the data (and semicontinuous data). It also claims that this is the only method that retains the original distribution of the data and gives plausible imputations for non-normally distributed data.
Vink, G., Frank, L.E., Pannekoek, J. and Van Buuren, S., 2014. Predictive mean matching imputation of semicontinuous variables. Statistica Neerlandica, 68(1), pp.61-90.
Although I am neither very familiar with STATA, it appears that this is what you are looking for.
Edit:
To clarify, this method corresponds to the case where data are missing at random (MAR). Data missing not at random (MNAR) is complicated, and options are more limited (in certain cases with no fix). Possible approaches require the distribution of missingness to be incorporated into the model. Selection models and pattern mixture models have been proposed for dealing with this situation - demonstrated nicely here.
In the case of MNAR I suggest conducting a sensitivity analysis to determine whether your data are robust to the MAR assumption.
-
$\begingroup$ Would this method of multiple imputation be appropriate for values not missing at random too? $\endgroup$ Commented Jul 28, 2018 at 12:29
-
$\begingroup$ Please see the edit. Hope this helps. $\endgroup$ Commented Jul 29, 2018 at 16:22