# Bayesian inference for multinomial distribution with asymmetric prior knowledge?

Suppose I will be getting some samples from a binomial distribution. One way to model my prior knowledge is with a Beta distribution with parameters $\alpha$ and $\beta$. As I understand it, this is equivalent to having seen "heads" $\alpha$ times in $\alpha + \beta$ trials. As such, a nice shortcut to doing the full-blown Bayesian inference is to use $\frac{h+\alpha}{n+\alpha+\beta}$ as my new mean for the probability of "heads" after having seen $h$ heads in $n$ trials.

Now suppose I have more than two states, so I will be getting some samples from a multinomial distribution. Suppose I want to use a Dirichlet distribution with parameter $\alpha$ as a prior. Again as a shortcut I can treat this as prior knowledge of event $i$'s probability as being equivalent to $\frac{\alpha_i}{\sum \alpha_j}$, and if I witness event $i$ $h$ times in $n$ trials my posterior for $i$ becomes $\frac{h + \alpha_i}{n + \sum \alpha_j}$.

Now in the binomial case, it works out that prior knowledge of "heads" occurring $\alpha$ times in $\alpha + \beta$ trials is equivalent to "tails" occurring $\beta$ times in $\alpha + \beta$ trials. Logically I don't believe I can have stronger knowledge of "heads" likelihood than of "tails." This gets more interesting with more than two outcomes though. If I have say a 6-sided die, I can imagine my prior knowledge of side "1" being equivalent to 10 ones in 50 trials and my prior knowledge of side "2" as being equivalent to 15 twos in 100 trials.

So after all of that introduction, my question is how I can properly model such asymmetric prior knowledge in the multinomial case? It seems as though if I'm not careful I can easily get illogical results due to total probability/likelihood not summing to 1. Is there some way I can still use the Dirichlet shortcut, or do I need to sacrifice this altogether and use some other prior distribution entirely?

Please forgive any confusion caused by potential abuses in notation or terminology above.

-
If I have say a 6-sided die, I can imagine my prior knowledge of side "1" being equivalent to 10 ones in 50 trials and my prior knowledge of side "2" as being equivalent to 15 twos in 100 trials. Is there a special reason why you do not want to unify your ' equivalent sample sizes' for "$1$" and "$2$" (namely $50$ and $100$) by using their least common multiple? In this case $\alpha_1 = 20/100$ and $\alpha_2 = 15/100$. – Zhubarb Nov 7 '13 at 11:44
It lets you represent asymmetric priors while ensuring $\sum_{i=1}^{i=6}\alpha_i = 1$ – Zhubarb Nov 7 '13 at 12:00
Finally, this paper by Kemp at al. is very relevant and fun to read. – Zhubarb Nov 7 '13 at 13:52
@Berkan I haven't read the paper yet (thanks though!), but the reason I didn't unify the sample sizes is because 20/100 is not the same prior as 10/50, and that fact is very important to the reason this question was asked. – Michael McGowan Nov 7 '13 at 14:43
Because you are more confident in the former due to a higher equiv. sample size? – Zhubarb Nov 7 '13 at 15:15