The expected value of the parameters of the multinomial distribution (taking into account the Dirichlet prior $D(\alpha)$ and the posterior Dirichlet-Multinomial) is:

$\pi_i = α_i+ x_i / \sum_{j} α_j+ x_j$

what are the other statistics and formulas of the multinomial distribution?

  • 1
    $\begingroup$ In both cases you use priors. MAP = mode of posterior distribution. $\endgroup$ – Tim Feb 21 '19 at 18:58
  • $\begingroup$ MAP is by definition mode of posterior distribution. "There will be difference" from what? $\endgroup$ – Tim Feb 21 '19 at 19:27
  • $\begingroup$ en.wikipedia.org/wiki/Mode_(statistics) $\endgroup$ – Tim Feb 21 '19 at 19:36
  • $\begingroup$ If I may suggest something, given your two questions, you'd probably gain more if you learned more/repeated the basic probability theory & statistics before studying more detailed material. Going directly to the complicated stuff may be very hard without understanding the basics. $\endgroup$ – Tim Feb 21 '19 at 19:40
  • $\begingroup$ you cannot compare the distribution with it's mode. It's like you asked what is the difference between age distribution in humans and the average height. $\endgroup$ – Tim Feb 21 '19 at 19:43

As said in previous answer to your question, in Dirichlet-multinomial model we assume Dirichlet prior for $\pi_1, \pi_2, \dots, \pi_k$ parameters of multinomial distribution, what leads to the following model

$$\begin{align} (x_1, x_2, \dots, x_k) &\sim \mathcal{M}(n, \, \pi_1, \pi_2, \dots, \pi_k) \\ (\pi_1, \pi_2, \dots, \pi_k) &\sim \mathcal{D}(\alpha_1, \alpha_2, \dots, \alpha_k) \end{align}$$

We estimate the parameters by applying Bayes theorem, and because the two Dirichlet is a conjugate prior for multinomial, we have closed formula for estimating the posterior distribution of $\pi_1, \pi_2, \dots, \pi_k$, that is Dirichlet with posterior parameters $\alpha_1+x_1, \alpha_2+x_2, \dots, \alpha_k+x_k$. If you want to get point estimates for the $\pi_1, \pi_2, \dots, \pi_k$ parameters, you can take mean of the distribution

$$ E(\pi_i) = \frac{\alpha_i + x_i}{\sum_{j=1}^k \alpha_j + x_j} $$

but you could as well look at other statistics of the distribution, like mode

$$ \mathrm{Mode}(\pi_i) = \frac{\alpha_i + x_i - 1}{\sum_{j=1}^k (\alpha_j + x_j -1)} $$

Notice that mode is defined for $\alpha_i > 1$, since otherwise mass of the distribution does not accumulate around single peak, as you can see on examples in this answer. Mode of posterior distribution is also known under the name of maximum a posteriori (MAP) estimate.

As a sidenote: The above formulas are very simple, but in many cases (not here) estimating the full posterior distribution is a hard problem. Often finding point estimate using MAP is much easier, since you can find MAP by optimization, rather then MCMC simulation for the full distribution. What follows, sometimes people directly estimate the mode (point estimate), without finding the full distribution.

You can be interested in other statistics of the posterior distribution as well (median, quantiles, etc.), depending on your needs.

| cite | improve this answer | |
  • $\begingroup$ @MosabShaheen As I said you should really start with learning basic concepts like estimator, or mode, before going to advanced topics. Otherwise this won't work. It is as if you went to university, to study literature, but didn't know how to read and write. There is infinitely many estimators. MAP is also a point estimator. Mode is the highest point of probability density function, but this goes beyond the question you asked. Also please notice that comments are not meant to asking follow up questions (ask new ones if you need) or lengthy discussions. $\endgroup$ – Tim Feb 22 '19 at 10:30
  • $\begingroup$ @MosabShaheen You asked "is expected value estimated using MAP?" So I did answer your question: expected value is not estimated by MAP, since MAP is mode of the posterior Dirichlet distribution. If you have follow up questions, like "how the formula for mode of Dirichlet distribution was derived", then ask a new question. At this point I will stop this discussion, since comments are really not meant for lengthy discussions. $\endgroup$ – Tim Feb 22 '19 at 10:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.