# Multiple Imputation (MI) for estimating desired a desired statistic but with missing data

Following [^Shafer] (page 4), and [^Austin et al.] (section "Analyses in the M imputed data sets"), which give a primer on Rubin's book for MI. 

Let 
- $Y$ be our data, and break it into $Y_{obs}$ (observed/existing), $Y_{mis}$ (missing subset). 
- $Q(Y)$ be the variable of interest, to be computed from our data. 
- We need that $Q(Y)$ is a normal random variable for the following all to hold. 

Method: 
- Find a way to simulate $Y_{mis}$, and generate $\{Y_{mis}^{(i)}\}_{i = 1}^m$, $m$ samples. 
- Compute $Q^i := Q(Y_{obs}, Y_{mis}^{(i)})$
- Compute $\bar{Q} := \sum_i Q^i / m$.  *This is our estimate!*

We can go further and get *t* statistic confidence intervals/hypothesis tests on how well $\bar{Q}$ approximates $Q$. 

Required is the variance of $Q$: 

0.  Recall Total Variance formula: $var(Q) = E_Y( var(Q|Y) ) + var_Y( E(Q|Y))$. 
1.  We have to *know* the $U:= var(Q|Y)$'s formula. 
2.  Compute $U^{i}:= var(Q^i|Y^i)$ 
3.  Estimate $E_Y( var(Q|Y) )$ by computing $\bar{U}:= \sum U^i/m$. 
	- *This is the first term of the total variance formula* 
	- This is called *average within-imputation variance*
4.  $\bar{Q}:= \sum(Q^i)/m $, our estimate of $E(Q|Y)$ is already computed. 
5. Compute $B:= (m-1)^{-1}\sum (Q^i-\bar{Q})^2$ to estimate $var( \bar{Q})$
	- This is called *between-imputation variance*
6. Finally, the desired estimate for $T:= Var(Q) = (1+(1/m))B + \bar{U}$ 

### **Questions here!**
**Questions on step 5:**
- Isn't $B$ the sample variance for $\{Q^i\}$, which should estimate $var(Q(Y)|Y)$ which is just $U$?
- I think what we really want is $var_Y( E(Q|Y)$, which is the *square of the sample error of the mean*, which is $B/m$. This follows [^StandardError]: 

> The standard error [of the mean] is, by definition, the standard deviation of $ \bar{x}$ which is simply the square root of the variance:
> $\sigma_{\bar {x}} = \sqrt { \frac{\sigma^{2}}{n} } = \frac{\sigma}{ \sqrt{n}}$ 

- [^Austin et al.] go on to say 

> When focusing on a single statistic, the Monte Carlo error can be computed as $\sqrt{B/M}$. 

So this seems to confirm my logic above. What am I missing here?


**Questions on step 6:**
- Why multiply $B$ by $(1+1/m)$?

-----
The paper's go on to say we can do Student T Test confidence intervals: 
1. $(Q-\bar{Q})/T \sim $ Student T statistic with $\nu = (m-1)[1 + \frac{\bar{U}}{(1+(1/m))B}]^2$ 
2. This means $\bar{Q} \pm AT$ is a $k$ confidence interval if $A$ is the two-sided t-value at $(1-k,  \nu)$. 

Question: If $m$ is sufficiently large (unsure, maybe 10K?) couldn't we use a z-score/Normal for our tests since Q(Y) normal?

**General Question** 
Can you check my understanding of why we cannot just use $ \sum(Q^i - \bar{Q})^2/(m-1)$ as the estimate of variance of $Q$.  Because if $Y_{mis}$ doesn't effect $Q$ at all, then we'd get variance of 0 with this computation. We need $U$ to account for the variance that comes from $Q$ on $Y_{obs}$. Is that right?


## References

[^Shafer]: https://www.sciencedirect.com/science/article/pii/S0828282X20311119 

[^Austin et al.]: https://www.sciencedirect.com/science/article/pii/S0828282X20311119

[^Fisher]: https://en.wikipedia.org/wiki/Fisher_transformation 

[^StandardError]: https://en.wikipedia.org/wiki/Standard_error#Standard_error_of_mean_versus_standard_deviation