12
$\begingroup$

I'm doing a project that involves replacing missing values in a set of data (first time doing this). This involves using two methods replacement by mean and replacement by median to fill in the missing values. There is not a lot of difference between the results of the minimum, median, maximum, mean and standard deviation of the data using both methods and I was wondering which method is better and how can I make a decision to which one is better using the results produced?

$\endgroup$
5
  • 3
    $\begingroup$ If you replace missings with means, naturally the mean is preserved. Ditto medians. Nor will the extremes change. The SDs will typically be reduced slightly, but it would be reduced greatly if you do this a lot. These are predictable consequences of what you do and not ipso facto indications that the method is good. $\endgroup$
    – Nick Cox
    Mar 27, 2015 at 15:54
  • 1
    $\begingroup$ Analysts plugging missing values (MVs) with automatic "solutions" like this aren't thinking through the consequences. It's just an easily implemented approach. This "solution" introduces as many problems as it solves since an otherwise typically smooth pdf ends up with a large spike at the plugged value, as a function of the number of MVs, of course. Model-based imputations are demonstrably superior and less biasing than any automated approach. @NickCox can't be ignorant of this, despite what his suggestion implies. $\endgroup$ Mar 18, 2017 at 17:20
  • 2
    $\begingroup$ :@DJohnson ... not ipso facto indications that the method is good. Not clear enough? $\endgroup$
    – Nick Cox
    Mar 18, 2017 at 19:46
  • $\begingroup$ Categorical values are generally imputed with the mode as it represents the value that is the most common for the given column. $\endgroup$ Jan 12, 2020 at 21:33
  • $\begingroup$ Replacement by mean or median --- or mode -- is in effect saying that you have no information on what a missing value might be. It is hard to know why imputation is though to help in that circumstance. Much hinges on whether the variable with missing values is regarded as a response or outcome to be predicted or as a predictor, and naturally it may have different roles for different purposes. What can make sense is to see whether the observations with missing values on this variable are systematically different on the variables that are known from the observations with non-missing values. $\endgroup$
    – Nick Cox
    Jun 4, 2020 at 8:39

3 Answers 3

14
$\begingroup$

It always depends on your data and your task.

If there is a dataset that have great outliers, I'll prefer median. E.x.: 99% of household income is below 100, and 1% is above 500.

On the other hand, if we work with wear of clothes that customers give to dry-cleaner (assuming that dry-cleaners' operators fill this field intuitively), I'll fill missings with mean value of wear.

It is better to start from data understanding and then this article will be helpful starting point.

$\endgroup$
2
  • $\begingroup$ The data I'm using can range from 0 to 1 and I've created histograms with limits of 0.1,0.2,0.3...to 1. Because I have many different limits and outliners would you say that the mean is best? $\endgroup$
    – Jake M-B
    Mar 27, 2015 at 15:10
  • $\begingroup$ @JakeM-B, it's hard give good advice, when I have not direct access to and history of the data. Often missing value in the data means that the value should be zero (or something else as default). On your place (if there is no great difference between mean and median), I would try both of them and check how it influence the outcome result. $\endgroup$ Mar 27, 2015 at 15:19
9
$\begingroup$

Imputation is a means to a goal, not the goal in itself. In some circumstances, replacing missing data might be the wrong thing to do. Make sure that you first pay attention to why your data are missing, as explained for example in the Missing data Wikipedia page, and that imputation is actually appropriate for answering the question your project seeks to answer.

If some assumptions are met (for example, if the probability of a variable having a missing value does not depend on the value itself, technically called "missing at random") and your study involves multiple variables, you might be better off using multiple imputation rather than replacements by means or medians. In multiple imputation, known values of all variables are used to provide several sets of estimates of the missing data. This approach can provide better estimates both of the underlying relations among the variables and of the reliability of your estimates. See questions on this site having the multiple-imputation tag for more information.

$\endgroup$
1
$\begingroup$

Imputing with the median is more robust than imputing with the mean, because it mitigates the effect of outliers. In practice though, both have comparable imputation results.

However, these two methods do not take into account potential dependencies between columns, which may contain relevant information to estimate missing values. More sophisticated algorithms like MissForest or MICE (both instances of the Iterative Imputer) or the kNNImputer provide much better imputation quality when mesured with the imputation RMSE. Here is a banchmark for data imputation methods: https://www.frontiersin.org/articles/10.3389/fdata.2021.693674/full

You can also find a nice introduction to these methods with ScikitLearn in Python here: https://scikit-learn.org/stable/modules/impute.html

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.