0
$\begingroup$

Missing values need to be imputed when performing logistic regression. Missing values are often imputed by the median. The mean is the BLUE (best linear unbiased estimator). But Median is robust to outliers and is not impacted by winsorization.

What are good statistical arguments to stick to median imputing over mean imputing apart from the procedural argumentation based on outliers and winsorization?

My feeling is that the median is a more robust estimator given a Non-Gaussian distribution and almost identical if Gaussian, but I don't know how to show this and if true.

And feel free also to convince me that mean is actually the better choice better.

Some thoughts so far:

  • Mean and median are both unbiased
  • As the sample size increases they will both approach the true mean of the distribution.
  • If the distribution is gaussian, it is expected that mean, median and mode will be the same in expectation.
  • If the distribution is lognormal, the median will be where the underlying normal distribution has its mean. Maybe this can be connected to logistic regression.
  • Median has a higher Breakdown Point than Mean. Breakdown Point measures how many observations would be needed to be transformed to outliers until the computed metric itself falls outside of the normal range.
  • Loosely speaking MSE is connected to mean and MAE is connected to Median. So maybe it is a question of what kind of error I am willing to except in particular what is my cost function.
$\endgroup$
4
  • 4
    $\begingroup$ "Missing values are often imputed by the median." What is the source of this claim? I also think that you should consider multiple imputation instead of single imputation. $\endgroup$
    – T.E.G.
    Commented Jun 6, 2023 at 13:08
  • $\begingroup$ It's the most common practice I have seen in credit scoring. But maybe I should have phrased it more conservatively, just what I have seen as best practice in multiple model developments. $\endgroup$
    – PalimPalim
    Commented Jun 6, 2023 at 13:17
  • 1
    $\begingroup$ Related question: stats.stackexchange.com/questions/143700/… $\endgroup$
    – T.E.G.
    Commented Jun 6, 2023 at 15:09
  • $\begingroup$ Thanks that is applicable, but I am hoping to find a statistically more rigorous answer which finds an answer why BLUE or similar might not be applicable. $\endgroup$
    – PalimPalim
    Commented Jun 6, 2023 at 15:53

2 Answers 2

4
$\begingroup$

What are good statistical arguments to stick to median imputing over mean imputing...

There are none.

...apart from the procedural argumentation based on outliers and winsorization?

The idea of a median as a "robust" location - robust to outliers - matters only for estimation and inference, but not imputation. Imputation cares about the whole probability model.

Imputing a "median" for a logistic analysis is total nonsense. If you think carefully about what you're saying, it is the same as "over half of my sample is a positive case, so I will impute my missing value as positive". If 5/9 responses are positive, then your complete sample will be 6/10 - biased by a factor of 5% from the complete case analysis - which is unbiased. The very least that could be said about median imputation is that you would impute an in-sample value, either 0 or 1, whereas mean imputation would be some fractional value. R barks a warning at you when you do this, but for no reason whatsoever. Logistic regression converges just fine with fractional values.

Unlike linear regression, imputing with prediction at the means will bias a logistic regression - surely not as badly as imputing a 0 or 1. This is a feature of non-collapsibility. But the overarching issue is anticonservative inference with mean imputation - you don't get good standard error estimates when you plug in the mean. Simulate the response, rather, and average up the estimates using Rubin's Rules or integrate over the likelihood to recover information from missingness.

$\endgroup$
2
$\begingroup$

As @T.E.G noted, single imputation is not the default.

Before thinking about imputation, it is also always important to think about the causal model at hand: is the data missing 'Completely at random (MCAR)', 'Missing at random (MAR)' (missing due to e.g. item nonresponse in surveys, imputation may be sensible here), and 'Missing not at random' (MNAR).

In the latter case, due to sample selection biases, no statistical imputation method alone can help to recover the true estimand. Causal assumptions are required here.

For a decent and brief write-up on causal inference/missing data/imputation, you can refer, for instance, to the following lecture notes:http://faculty.washington.edu/yenchic/19A_stat535/Lec12_causal_missing.pdf. The part on MNAR is a bit outdated though, refer to the causal inference + sample selection bias literature here.

$\endgroup$
1
  • $\begingroup$ Interestingly, they are also suggesting median for MCAR "Under MCAR, we can completely ignore the data with missing values and just use the sample median as an estimate of mY ." $\endgroup$
    – PalimPalim
    Commented Jun 21, 2023 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.