3 edited tags; edited title

# using Using regression for imputing missing data

2 added 2110 characters in body

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.

EDIT 1:

To give some technical/theoretical background, in section 3.2.1 of the book "Flexible Imputation of Missing Data":

For univariate $$Y$$ we write lowercase $$y$$ for $$Y$$ . Any predictors in the imputation model are collected in $$X$$. Symbol $$X_{obs}$$ indicates the subset of $$n_1$$ rows of $$X$$ for which $$y$$ is observed, and $$X_{mis}$$ is the complementing subset of n 0 rows of $$X$$ for which $$y$$ is missing. The vector containing the $$n_1$$ observed data in $$y$$ is denoted by $$y_{obs}$$ , and the vector of $$n_0$$ imputed values in $$y$$ is indicated by $$\dot{y}$$. This section reviews four different ways of creating imputations under the normal linear model. The four methods are:

1. Predict. $$\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}}$$ , where $$\hat{\beta_{0}}$$ and $$\hat{\beta_{1}}$$ are least squares estimates calculated from the observed data. Section 1.3.4 named this regression imputation. In mice this method is available as "norm.predict".

2. Predict + noise. $$\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}} + \dot{\epsilon}$$, where $$\dot{\epsilon}$$ is randomly drawn from the normal distribution as $$\dot{\epsilon} \sim N(0, \hat{\sigma}^2)$$. Section 1.3.5 named this stochastic regression imputation. In mice this method is available as "norm.nob".

3. Bayesian multiple imputation. $$\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$$, where $$\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$$ and $$\dot\beta_{0}$$ , $$\dot\beta_{1}$$ and $$\dot\sigma$$ are random draws from their posterior distribution, given the data. Section 3.1.3 named this “predict + noise + parameters uncertainty.” The method is available as "norm".

4. Bootstrap multiple imputation. $$\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$$, and where $$\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$$ and $$\dot\beta_{0}$$ , $$\dot\beta_{1}$$ and $$\dot\sigma$$ are the least squares estimates calculated from a bootstrap sample taken from the observed data. This is an alternative way to implement “predict + noise + parameters uncertainty.” The method is available as "norm.boot".

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.

EDIT 1:

To give some technical/theoretical background, in section 3.2.1 of the book "Flexible Imputation of Missing Data":

For univariate $$Y$$ we write lowercase $$y$$ for $$Y$$ . Any predictors in the imputation model are collected in $$X$$. Symbol $$X_{obs}$$ indicates the subset of $$n_1$$ rows of $$X$$ for which $$y$$ is observed, and $$X_{mis}$$ is the complementing subset of n 0 rows of $$X$$ for which $$y$$ is missing. The vector containing the $$n_1$$ observed data in $$y$$ is denoted by $$y_{obs}$$ , and the vector of $$n_0$$ imputed values in $$y$$ is indicated by $$\dot{y}$$. This section reviews four different ways of creating imputations under the normal linear model. The four methods are:

1. Predict. $$\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}}$$ , where $$\hat{\beta_{0}}$$ and $$\hat{\beta_{1}}$$ are least squares estimates calculated from the observed data. Section 1.3.4 named this regression imputation. In mice this method is available as "norm.predict".

2. Predict + noise. $$\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}} + \dot{\epsilon}$$, where $$\dot{\epsilon}$$ is randomly drawn from the normal distribution as $$\dot{\epsilon} \sim N(0, \hat{\sigma}^2)$$. Section 1.3.5 named this stochastic regression imputation. In mice this method is available as "norm.nob".

3. Bayesian multiple imputation. $$\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$$, where $$\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$$ and $$\dot\beta_{0}$$ , $$\dot\beta_{1}$$ and $$\dot\sigma$$ are random draws from their posterior distribution, given the data. Section 3.1.3 named this “predict + noise + parameters uncertainty.” The method is available as "norm".

4. Bootstrap multiple imputation. $$\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$$, and where $$\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$$ and $$\dot\beta_{0}$$ , $$\dot\beta_{1}$$ and $$\dot\sigma$$ are the least squares estimates calculated from a bootstrap sample taken from the observed data. This is an alternative way to implement “predict + noise + parameters uncertainty.” The method is available as "norm.boot".

1

# using regression for missing data

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.