multinomial distribution aggregation property Suppose we have multinomial distribution in which we have 4 categories, and each one is associated with a probability of being selected, say $\theta_i$,  $i=1,..,4$.
And I know for sure that $\theta_1=0.1, \theta_2=0.2$. Now I want to Bayesain inference the unknown $\theta_3$ (the $\theta_4$ can be deduced since $\sum \theta_i=1$) from evidence, say $n_i$ observstions for each category $i$.
The problem is, I see two ways to do this:


*

*As the one here :multinomial model with some certain parameters. In other words, discard the evidence of $n_1, n_2$ and rescale the $\theta_3,\theta_4$ as a binomial distribution then do the Bayesian inference only with evidence of $n_3,n_4$.

*Use marginal distribution of $\theta_3$, i.e. define a new binomial distribution with probabilities $p_3=\theta_3$ and $1-p_3=0.7-\theta_3$. Then do Bayesian inference for $p_3$ with evidence $n_3$ and $(n_1+n_2+n_4)$.
I guess these two ways are both correct and will reach the same results for a $E(\theta_3|data)$, am I right? just to confirm...
PS, once I built a new binomial distribution, to do the Bayesian, I do not have to use Beta distribution as my prior for $p_3$, right? I can use whatever I want, e.g. a two-point one..
Thanks guys.
 A: In general, these are not equivalent. Intuitively, if we assume $p_1$, $p_2$ are known, we should have less posterior uncertainty about $p_3$ and $p_4$. Another way to look at it is this: given that we know $p_1$ and $p_2$, we also know $p_3+p_4$. The only free parameter is equivalent to $p_3/p_4$, so $n_1$ and $n_2$ are not useful to us in any way.
Here is a little simulation bearing that out. (Note that I used Jeffrey's prior, $\operatorname{Beta}(0.5,0.5)$ and plots are facetted by sample size and method.)
The first plot shows that the posterior expectation has a similar sampling distribution and the second shows that the posterior standard deviation is larger using the marginal method as compared with the ignoring method.
library(MCMCpack)
library(plyr)
library(ggplot2)

results <- ldply(c(10, 100, 1000), function(sz){

  n <- rmultinom(1000, size=sz, prob = c(.1,.2,.3,.4))

  jeffreys_prior <- c(.5,.5)

  E_post <- function(f, s, prior){
    (s + prior[2])/(s+f+sum(prior))
  }
  SD_post <- function(f, s, prior){
    sqrt((f + prior[1])*(s+prior[2])/((s+f+sum(prior))^2*(s+f+sum(prior)+1)))
  }

  marginal_results = adply(n, 2, function(dat){
    data.frame(mean =  E_post(sum(dat[c(1,2,4)]), dat[3], jeffreys_prior),
               sd   = SD_post(sum(dat[c(1,2,4)]), dat[3], jeffreys_prior),
               method = "marginal")
  })

  ignoring_results = adply(n, 2, function(dat){
    data.frame(mean =  E_post(dat[4], dat[3], jeffreys_prior) * .7,
                 sd = SD_post(sum(dat[c(1,2,4)]), dat[3], jeffreys_prior) * .7,
              method= "ignoring")
  })
  result = rbind(marginal_results, ignoring_results)
  result$size = sz
  result
})
ggplot(results, aes(x=mean)) + geom_histogram(bins=20) + facet_grid(method~size, scales = "free_x") + theme_bw()
ggplot(results, aes(x=sd)) + geom_histogram() + facet_grid(method~size) + theme_bw()



