# Multiple Imputation (MI) for estimating desired a desired statistic but with missing data

Following ^Shafer (page 4), and ^Austin et al. (section "Analyses in the M imputed data sets"), which give a primer on Rubin's book for MI.

Let

• $$Y$$ be our data, and break it into $$Y_{obs}$$ (observed/existing), $$Y_{mis}$$ (missing subset).
• $$Q(Y)$$ be the variable of interest, to be computed from our data.
• We need that $$Q(Y)$$ is a normal random variable for the following all to hold.

Method:

• Find a way to simulate $$Y_{mis}$$, and generate $$\{Y_{mis}^{(i)}\}_{i = 1}^m$$, $$m$$ samples.
• Compute $$Q^i := Q(Y_{obs}, Y_{mis}^{(i)})$$
• Compute $$\bar{Q} := \sum_i Q^i / m$$. This is our estimate!

We can go further and get t statistic confidence intervals/hypothesis tests on how well $$\bar{Q}$$ approximates $$Q$$.

Required is the variance of $$Q$$:

1. Recall Total Variance formula: $$var(Q) = E_Y( var(Q|Y) ) + var_Y( E(Q|Y))$$.
2. We have to know the $$U:= var(Q|Y)$$'s formula.
3. Compute $$U^{i}:= var(Q^i|Y^i)$$
4. Estimate $$E_Y( var(Q|Y) )$$ by computing $$\bar{U}:= \sum U^i/m$$.
• This is the first term of the total variance formula
• This is called average within-imputation variance
5. $$\bar{Q}:= \sum(Q^i)/m$$, our estimate of $$E(Q|Y)$$ is already computed.
6. Compute $$B:= (m-1)^{-1}\sum (Q^i-\bar{Q})^2$$ to estimate $$var( \bar{Q})$$
• This is called between-imputation variance
7. Finally, the desired estimate for $$T:= Var(Q) = (1+(1/m))B + \bar{U}$$

### Questions here!

Questions on step 5:

• Isn't $$B$$ the sample variance for $$\{Q^i\}$$, which should estimate $$var(Q(Y)|Y)$$ which is just $$U$$?
• I think what we really want when we say $$var_Y( E(Q|Y)$$ is the square of the sample error of the mean, which is $$B/m$$. This follows ^StandardError:

The standard error [of the mean] is, by definition, the standard deviation of $$\bar{x}$$ which is simply the square root of the variance: $$\sigma_{\bar {x}} = \sqrt { \frac{\sigma^{2}}{n} } = \frac{\sigma}{ \sqrt{n}}$$

When focusing on a single statistic, the Monte Carlo error can be computed as $$\sqrt{B/M}$$.

So this seems to confirm my logic above. If so, then is between-imputation variance $$B$$ or $$B/M$$ or $$(1+1/m)B$$ ?

Questions on step 6:

• It seems to me that the total variance formula says $$var(Q) = B/M + \bar{U}$$. Am I right?
• If so, why does the formula in these papers multiply $$B$$ by $$(1+1/m)$$ and use $$var(Q) = B + B/m + \bar{U}$$?

The paper's go on to say we can do Student T Test confidence intervals:

1. $$(Q-\bar{Q})/T \sim$$ Student T statistic with $$\nu = (m-1)[1 + \frac{\bar{U}}{(1+(1/m))B}]^2$$
2. This means $$\bar{Q} \pm AT$$ is a $$k$$ confidence interval if $$A$$ is the two-sided t-value at $$(1-k, \nu)$$.

Question: If $$m$$ is sufficiently large (unsure, maybe 10K?) couldn't we use a z-score/Normal for our tests since Q(Y) normal?

General Question Can you check my understanding of why we cannot just use $$\sum(Q^i - \bar{Q})^2/(m-1)$$ as the estimate of variance of $$Q$$. Because if $$Y_{mis}$$ doesn't effect $$Q$$ at all, then we'd get variance of 0 with this computation. We need $$U$$ to account for the variance that comes from $$Q$$ on $$Y_{obs}$$. Is that right?

The total variance formula you would like to use is based on an infinitely large number of imputations, with $$\bar U_{\infty}$$ and $$B_{\infty}$$ the within- and between-imputation variance. The estimate of the within-imputation variance $$\bar U$$ based on a finite number $$m$$ of imputations is unbiased, but there's a bias if you substitute the finite-imputation $$B$$ for $$B_{\infty}$$. Quoting Section 2.3.2 of van Buuren's Flexible Imputation of Missing Data:

It is tempting to conclude that the total variance $$T$$ is equal to the sum of $$\bar U$$ and $$B$$, but that would be incorrect. We need to incorporate the fact that $$\bar Q$$ itself is estimated using finite $$m$$, and thus only approximates $$Q_{\infty}$$. Rubin shows that the contribution to the variance of this factor is systematic and equal to $$B_{\infty}/m$$.

With $$B$$ providing an approximation to $$B_{\infty}$$, you get the 3rd contribution to the total variance, $$B/m$$, that you are questioning.

• This is a great link. thanks for this reference. Mar 30, 2022 at 1:08

The Question as posed has an error as stated:

1. Compute $$𝐵:=(𝑚−1)^{−1}\sum(𝑄^{i}−\bar{𝑄})^2$$ to estimate $$var(\bar{𝑄})$$

should be Compute $$𝐵:=(𝑚−1)^{−1}\sum(𝑄^{i}−\bar{𝑄})^2$$ to estimate the sample variance of $$\{Q^i\}$$ (Not variance of $$\bar{Q}$$). This error may be leading to some of the confusion. The goal is not to compute variance of $$\bar{Q}$$.

This answers the "Questions on step 5":

• for the first one is that yes $$B$$ is the sample variance for $$\{Q^i\}$$, which is different in general than $$U=mean_i ( var(Q^{i}|Y^i))$$.
• for the second question, yes $$B/m$$ is the empirical estimate for the variance of $$\bar{Q}$$ see Wikipedia page on Standard Error of the Mean vs Standard Deviation, but that is not what we are estimating.

Moving to the answers for the "Questions on step 6":

• it follows that the variance of $$Q$$ is $$B + U$$ from the total variance formula see Wikipedia page
• the answer by EdM shows that the empirical estimate of the total variance is $$U + B + B/m$$, which answers the second question on step 6.

Moving to the final question, looking at a t-table we see that for sufficiently large degrees of freedom ($$\nu >= 1000$$, say) it is the same as a z-table. From the computation for imputation t-test, for large $$m$$, this is the case. So the answer to the final question is "yes".