5
$\begingroup$

I want to impute missing values of a dataset for machine learning (knn imputation). Is it better to scale and center the data before the imputation or afterwards?

Since the scaling and centering might rely on min and max values, in the first case the subsequent imputation might add new max / min values and tamper the scaled/centered data.

However, the imputation process might also profit from a scaled and centered dataset.

What do you think is better, and why?

$\endgroup$
  • 1
    $\begingroup$ It is better to impute data during the machine learning process using EM algorithm. This way you are avoiding forcing a topology that will be inferior to the task at hand. $\endgroup$ – Cowboy Trader Feb 18 '15 at 18:00
2
$\begingroup$

Presumably, if you really need to center & scale the data, that should be done after imputation, as the imputation could influence on the correct center and scale to use!

Generally, the imputation should be the very first step in any analysis you do.

EDIT answer to comment by @Inon:

You say that imputation should preserve center & scale, and also standardization. Why? If the missing values truly are missing at random, maybe it does not matter much, but generally missingness might depend on other observed variables, and then estimates of mean and scale could be skewed by this pattern in the missingness. Imputation (better multiple imputation) is a way to fight this skewing. But if you do imputation after scaling, you just preserve the bias introduced by the missingness mechanism. Imputation is meant to fight this, and doing imputation after scaling just defeats this.

$\endgroup$
  • $\begingroup$ I'm not sure I understand your reasoning. We don't want imputed values to affect the center and scale of the data set. So wouldn't it be easier to ignore the missing values before they are replaced with valid values? It seems to me that imputation should follow enumeration of categorical features, standardization (of mean and variance) and vectorization (where relevant), and precede censoring, binning, and other numerical transformations. What do you think? $\endgroup$ – Inon Nov 9 '16 at 23:30
0
$\begingroup$

It really depends on the Imputation technique being used. For example if we Impute using distance based measure (eg. KNN), then it is recommended to first standardize the data and then Impute. That is because lower magnitude values converge faster.

One idea could be using preprocess function from caret package. When you use method = knnImpute, it first center and scale the data before imputation.

preProcValues <- preProcess(data, method = c("knnImpute","center","scale"))

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.