Using regression for imputing missing data

Question

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.

EDIT 1:

To give some technical/theoretical background, in section 3.2.1 of the book "Flexible Imputation of Missing Data":

For univariate $Y$ we write lowercase $y$ for $Y$ . Any predictors in the imputation model are collected in $X$. Symbol $X_{obs}$ indicates the subset of $n_1$ rows of $X$ for which $y$ is observed, and $X_{mis}$ is the complementing subset of n 0 rows of $X$ for which $y$ is missing. The vector containing the $n_1$ observed data in $y$ is denoted by $y_{obs}$ , and the vector of $n_0$ imputed values in $y$ is indicated by $\dot{y}$. This section reviews four different ways of creating imputations under the normal linear model. The four methods are:

1. Predict. $\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}}$ , where $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$ are least squares estimates calculated from the observed data. Section 1.3.4 named this regression imputation. In mice this method is available as "norm.predict".

2. Predict + noise. $\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}} + \dot{\epsilon}$, where $\dot{\epsilon}$ is randomly drawn from the normal distribution as $\dot{\epsilon} \sim N(0, \hat{\sigma}^2)$. Section 1.3.5 named this stochastic regression imputation. In mice this method is available as "norm.nob".

3. Bayesian multiple imputation. $\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$, where $\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$ and $\dot\beta_{0}$ , $\dot\beta_{1}$ and $\dot\sigma$ are random draws from their posterior distribution, given the data. Section 3.1.3 named this “predict + noise + parameters uncertainty.” The method is available as "norm".

4. Bootstrap multiple imputation. $\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$, and where $\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$ and $\dot\beta_{0}$ , $\dot\beta_{1}$ and $\dot\sigma$ are the least squares estimates calculated from a bootstrap sample taken from the observed data. This is an alternative way to implement “predict + noise + parameters uncertainty.” The method is available as "norm.boot".

Answer 1

I believe that the core idea here is that you are filling in the missing bits, not deriving the entire value. If you are working with something more strictly decisive like a decision trees, it prevents the "missing" values from becoming a class of their own. If you can PERFECTLY predict f2 from only f1 in all cases then sure it might not be a valuable feature since it is too correlated, but if you are missing values from each on different samples then together you can produce a complete picture.

However, if you don't do the gap fill then a decision tree or a neuron applied to the value would get a completely different output / y_hat and you are at risk of weakening a good correlation between the features and the result, especially if it is a representative example on a minority class.