# multinomial distribution aggregation property

Suppose we have multinomial distribution in which we have 4 categories, and each one is associated with a probability of being selected, say $\theta_i$, $i=1,..,4$.

And I know for sure that $\theta_1=0.1, \theta_2=0.2$. Now I want to Bayesain inference the unknown $\theta_3$ (the $\theta_4$ can be deduced since $\sum \theta_i=1$) from evidence, say $n_i$ observstions for each category $i$.

The problem is, I see two ways to do this:

1. As the one here :multinomial model with some certain parameters. In other words, discard the evidence of $n_1, n_2$ and rescale the $\theta_3,\theta_4$ as a binomial distribution then do the Bayesian inference only with evidence of $n_3,n_4$.

2. Use marginal distribution of $\theta_3$, i.e. define a new binomial distribution with probabilities $p_3=\theta_3$ and $1-p_3=0.7-\theta_3$. Then do Bayesian inference for $p_3$ with evidence $n_3$ and $(n_1+n_2+n_4)$.

I guess these two ways are both correct and will reach the same results for a $E(\theta_3|data)$, am I right? just to confirm...

PS, once I built a new binomial distribution, to do the Bayesian, I do not have to use Beta distribution as my prior for $p_3$, right? I can use whatever I want, e.g. a two-point one..

Thanks guys.

• Regarding your PS: in general you're free to use any prior you want. In practice, certain priors may not result in an easy-to-sample posterior. Point priors, however, shouldn't pose an issue. – zipzapboing Jun 8 '18 at 23:50

In general, these are not equivalent. Intuitively, if we assume $p_1$, $p_2$ are known, we should have less posterior uncertainty about $p_3$ and $p_4$. Another way to look at it is this: given that we know $p_1$ and $p_2$, we also know $p_3+p_4$. The only free parameter is equivalent to $p_3/p_4$, so $n_1$ and $n_2$ are not useful to us in any way.

Here is a little simulation bearing that out. (Note that I used Jeffrey's prior, $\operatorname{Beta}(0.5,0.5)$ and plots are facetted by sample size and method.)

The first plot shows that the posterior expectation has a similar sampling distribution and the second shows that the posterior standard deviation is larger using the marginal method as compared with the ignoring method.

library(MCMCpack)
library(plyr)
library(ggplot2)

results <- ldply(c(10, 100, 1000), function(sz){

n <- rmultinom(1000, size=sz, prob = c(.1,.2,.3,.4))

jeffreys_prior <- c(.5,.5)

E_post <- function(f, s, prior){
(s + prior)/(s+f+sum(prior))
}
SD_post <- function(f, s, prior){
sqrt((f + prior)*(s+prior)/((s+f+sum(prior))^2*(s+f+sum(prior)+1)))
}

data.frame(mean =  E_post(sum(dat[c(1,2,4)]), dat, jeffreys_prior),
sd   = SD_post(sum(dat[c(1,2,4)]), dat, jeffreys_prior),
method = "marginal")
})