Trouble with imputed data set

Question

1) I had a dataset with missing data for baseline variables and outcome variables. Through multiple imputation in SPSS (10 imputations, 50 iterations, PMM for scale variables) I imputed the missing data for the baseline variables. When I analyse the data (i.e. only the 10 imputation data sets) and use the independent sample t-test for a continuous variable results are pooled. This includes mean and p-value, but not standard deviation. Is it correct that I can take the average of the 10 standard deviations calculated for the imputed data sets to calculate the pooled standard deviation?

2) The pooled p-value from the independent t-test on 10 imputation data sets in SPSS is not the average of the 10 p-values that are calculated: am I correct? As I believe a correction is made for the fact that the p-value is based on imputed data?

3) I want to impute the missing data for the outcome variables as well. I did not include those variables in the first imputation process since one of the outcome variables is affected by the value of one of the baseline variables for which data was missing. So I figured that I would first impute the baseline variable, create a 'complete' data set for it and than impute the outcome variables using that data and the other baseline variables. Is it actually possible to impute a second time? And how would I do that? Now (after the first imputation process) I have a dataset which is 11 times as large as originally (original data set + 10 x imputed data set). Do I use all 10 imputation data sets to run the imputation process again which would result in another tenfold of that data set (so 100 fold of my original data set size)? And is it true this would lead to a larger standard deviation for scale variables?

Answer 1

1. Taking the average of the 10 standard deviations is not correct. Following Rubin's rules for multiple imputation, you can use the following formula to get the pooled standard deviation

$$Var(\bar{\beta}) = m^{-1} \sum_{k=1}^m Var(\hat{\beta_k}) + (1+m^{-1})(m-1)^{-1} \sum_{k=1}^m (\hat{\beta_k} - \bar{\beta})^2$$ where $\hat{\beta}_k$ is the estimated point estimate for imputation indexed by $k$, $Var(\hat{\beta})$ is the variance of the point estimates for a single imputation, and $$\bar{\beta} = m^{-1} \sum_{k=1}^m \hat{\beta_k}$$ for $m$ imputations (10 in your case). For the standard deviation, you would $\sqrt{Var(\bar{\beta})}$. The above calculation is important since it accounts for the variation between imputations and within imputations, which the average of the standard deviations from imputations does not.

2. Correct. You would use $\bar{\beta}$ and $Var(\bar{\beta})$ to calculate the p-value.

3. I believe you should include the outcome is the first model. It should be possible with SPSS to indicate the order in which the variables are imputed. You would want the outcome to be imputed last. My knowledge of SPSS is limited, so I can't help on this point as much. Hopefully someone else can comment on this part.

Answer 2

1. I've never seen Rubin, Little, or Schafer explicitly address pooling of standard deviations. I would tend to agree with Jeremy that it's probably ok, though one possible approach that I'm sure one could support would be to estimate variances, average them, and take the square root. You could actually do this using the MIXED procedure with just a fixed intercept. The residual variance is the variance measure of interest, and you would get a pooled value for this.

2. Right. See the equations in the statistical algorithms for multiple imputation pooling for the precise formulas.

3. The MULTIPLE IMPUTATION procedure won't work on data that's already got imputations (if the Imputation_ variable exists, it draws an error). I'm also not sure why you didn't do the full estimation at once, as that would be the typical way to do this.