I have to choose between median or mean imputation to handle missing values. I feel median imputation will work better because it is a number that is already present in the data set and is less susceptible to outlier errors as compared to mean imputation.
What might be the disadvantages of median imputation though?
These are not appropriate for computing missing data - consider the case of heteroskedasticity in the data - neither of these approaches would work if their were 'weird' or idiosyncratic values in your data. In fact it would be more damaging (ie less accurate) to use mean or median replacement in this case
if youre familiar with R, you could check out the MI package (my fave) or mice. This essentially runs a series of chained (ie bayesian) regressions on the data until some convergence criteria
other options are expectation maximization (subject to overfitting problems IMO) and Hotdeck imputation
check out these resources for more explanation about why mean/median replacement is generally a bad idea
Rubin, D. B. (1996). Multiple imputation after 18+ years. Journal of the American Statistical Association, 91(434):473–489.
Schafer, J. L. (1999). Multiple imputation: a primer. Statistical Methods in Medical Research, 8:3–15.
It depends on some factors. Using mean or median is not always the key to imputing missing values. I would agree that certainly mean and median imputation is the most famous and used method when it comes to handling missing data. However, there are other ways to do that. First of all, you do not want to change the distribution of the data. You have to place values so that the variance is not much. To ensure you are doing it right, you can look at the KDE plots before and after the imputation or overlap them. The variance will be clear. You can also use other techniques, such as 𝑹𝒂𝒏𝒅𝒐𝒎 𝑺𝒂𝒎𝒑𝒍𝒆 𝑰𝒎𝒑𝒖𝒕𝒂𝒕𝒊𝒐𝒏, 𝑪𝒂𝒑𝒕𝒖𝒓𝒊𝒏𝒈 𝑵𝒂𝑵 𝒗𝒂𝒍𝒖𝒆𝒔 𝒘𝒊𝒕𝒉 𝒏𝒆𝒘 𝒇𝒆𝒂𝒕𝒖𝒓𝒆𝒔, 𝑬𝒏𝒅 𝒐𝒇 𝒅𝒊𝒔𝒕𝒓𝒊𝒃𝒖𝒕𝒊𝒐𝒏 𝒊𝒎𝒑𝒖𝒕𝒂𝒕𝒊𝒐𝒏, 𝑨𝒓𝒃𝒊𝒕𝒓𝒂𝒓𝒚 𝑽𝒂𝒍𝒖𝒆 𝑰𝒎𝒑𝒖𝒕𝒂𝒕𝒊𝒐𝒏. Some of the techniques also record the importance of the missing data. you Can learn about these in the following link https://www.linkedin.com/feed/update/urn:li:activity:6958745603480698880/
You can also find the codes in https://github.com/protikmostafa083/Machine-Learning-Workbook/blob/main/Feature%20Engineering/Feature_Engineering_Handling_Missing_Values.ipynb
You can use an algorithm that is robust to missing values, such as k-NN, random forest, Naive Bayes etc.
