# multivariate dirichlet for multiple imputation

I dealing with 3 covariates {x1, x2, x3} all three are discrete and contain missing data.

                   x1  | x2| x3 |
--------------
3   | 2 |  . |
.   | 1 |  3 |
1   | . |  2 |
3   | 2 |  . |
2   | . |  1 |
.   | 3 |  1 |
--------------


I am exploring the option of using multiple imputation. I plan on generating observations using joint multivariate dirichlet distribution. Am i heading in the right direction ? Please advise.

Below is my whole dataset

/********************** pi***************************/

y   x1  x2  x3  Freq    pi
0   1   1   1   15  0.02078
1   1   1   1   7   0.0097
0   2   1   1   13  0.01801
1   2   1   1   9   0.01247
0   3   1   1   10  0.01385
1   3   1   1   13  0.01801
0   1   2   1   16  0.02216
1   1   2   1   11  0.01524
0   2   2   1   14  0.01939
1   2   2   1   18  0.02493
0   3   2   1   6   0.00831
1   3   2   1   26  0.03601
0   1   3   1   13  0.01801
1   1   3   1   16  0.02216
0   2   3   1   18  0.02493
1   2   3   1   16  0.02216
0   3   3   1   17  0.02355
1   3   3   1   15  0.02078
0   1   1   2   12  0.01662
1   1   1   2   18  0.02493
0   2   1   2   8   0.01108
1   2   1   2   17  0.02355
0   3   1   2   16  0.02216
1   3   1   2   11  0.01524
0   1   2   2   17  0.02355
1   1   2   2   10  0.01385
0   2   2   2   8   0.01108
1   2   2   2   14  0.01939
0   3   2   2   17  0.02355
1   3   2   2   11  0.01524
0   1   3   2   11  0.01524
1   1   3   2   13  0.01801
0   2   3   2   16  0.02216
1   2   3   2   23  0.03186
0   3   3   2   11  0.01524
1   3   3   2   21  0.02909
0   1   1   3   15  0.02078
1   1   1   3   8   0.01108
0   2   1   3   12  0.01662
1   2   1   3   11  0.01524
0   3   1   3   10  0.01385
1   3   1   3   11  0.01524
0   1   2   3   8   0.01108
1   1   2   3   13  0.01801
0   2   2   3   23  0.03186
1   2   2   3   11  0.01524
0   3   2   3   7   0.0097
1   3   2   3   14  0.01939
0   1   3   3   8   0.01108
1   1   3   3   5   0.00693
0   2   3   3   8   0.01108
1   2   3   3   16  0.02216
0   3   3   3   16  0.02216
1   3   3   3   19  0.02632

• Can you give more detail? Do $x_1, x_2, x_3$ all take values in $\{1, 2, 3\}$? It looks like in all your examples that each of $\{1, 2, 3\}$ might occur exactly once; is it possible, for example, to observe $(x_1, x_2, x_3) = (3, 3, 3)$? – guy Aug 21 '14 at 2:49
• Guy, x1,x2,x3 can take any values, {1,1,1},{1,1,2}, {1,2,1}, {1,1,3}, {1,3,1},{2,1,1},{2,1,2},{2,2,2}, {2,2,1},{2,1,3},.....any combination of these. – Tyrone Williams Aug 21 '14 at 2:52
• Is this your whole dataset? – guy Aug 21 '14 at 3:02
• guy I just posted the whole dataset – Tyrone Williams Aug 21 '14 at 6:14

## 1 Answer

One strategy that seems reasonable off the top of my head is to let $\pi_{ijk} = \Pr(X_1 = i, X_2 = j, X_3 = k \mid \pi)$ and then put a standard Dirichlet prior on $\mbox{Vec}(\pi) = (\pi_{111}, \pi_{112}, \ldots, \pi_{333})$. Then, for a fixed value of $\pi$, one can impute according to (for example) $$\Pr(X_3 = c \mid X_1 = a, X_2 = b, \pi) = \frac{\pi_{abc}}{\pi_{ab1} + \pi_{ab2} + \pi_{ab3}},$$ and $$\Pr(X_1 = a, X_2 = b \mid X_3 = c, \pi) = \frac{\pi_{abc}}{\sum_{a' = 1}^3 \sum_{b' = 1} ^ 3 \pi_{a' b' c}}.$$ These are just examples of imputations you might have to do when you are filling in $\mathbf X$. Once $\mathbf X$ is filled in, you can then update $\pi$ from the conjugate update of the Dirichlet distribution.

This approach requires having some complete data, preferable at every combination. With $1000$ observations and $27$ categories, this doesn't seem like a guarantee, but I think this is the best you can do without making any strong assumptions. There are other modelling strategies beyond this Dirichlet prior though; for example, one could use a log-linear model with some assumptions about higher order interactions to reduce the number of parameters.