# Difficulties in computing the derivatives of the Dirichlet distribution

I need to compute the first derivatives of the Dirichlet distribution, defined in the following way:

$$r(P; \pi, \rho) = \frac{\Gamma(c)}{\prod_{i=1}^{k} \Gamma(c \pi_i)} \cdot \prod_{i=1}^{k} P_i^{c\pi_i - 1},$$

where $$c=\rho^{-2}(1-\rho^2) = \sum_{i=1}^{k} \alpha_i$$. Now I need to compute the first derivative of the log likelihood with respect to $$\pi_i$$ and $$\rho$$ but I am finding myself in having a hard time. I defined the log likelihood as: $$\log\Gamma(c) - \sum_{i=1}^{k} \log\Gamma(c\pi_i) + \sum_{i=1}^{k}(c\pi_i-1) \log P_i,$$ and I hope it is correct. But then, I don't get how I can compute the derivatives, in particular the one with respect to $$\pi_i$$ because I have the summation with respect to i and a digamma function. Can somebody show it to me? just the derivative with respect to $$\pi_i$$ of the second and third term of the log likelihood.

And also: when I have the log of a gamma function I know that the derivative is the digamma function. But do I have to multiply the digamma function for the derivative of the argument of the digamma function (like chain rule of the derivatives)?

Ok, thanks a lot! I still have a doubt about the digamma and trigamma function. I need to compute the Fisher information and so after having computed the first derivative I compute also the second derivative which is actually equal to $$-c^2 * \psi'(c\pi_i)$$.

I have to compute the $$E_P[-d^2l/d\pi_i^2]$$ and I am given the result of this which is equal to $$c^2[\psi'(c\pi_i) + \psi'(c\pi_k)]$$ for $$i = 1,...,k-1$$. But how can I get this result? I don't get why this is the result... from where they took $$\psi'(c\pi_k)$$?

And also which is the expectation of a digamma and trigamma function? Thanks a lot again if you can help me!

• Is $\pi$ a vector with component $\pi_i$? Mar 14 at 12:53
• Yes exactly, with k elements
– Bibi
Mar 14 at 13:40

You are confusing yourself here by failing to recognise that the index of summation $$i$$ is just an index, not a variable in the equation. Consequently, if you expand out the sum, you get a form that does not have $$i$$ in it:

$$\sum_{i=1}^k f(\pi_i) = f(\pi_1) + \cdots + f(\pi_k).$$

To avoid confusion, it is good practice not to use the same index for your derivative and for your summation index. Taking the partial derivative with respect to an arbitrary element $$\pi_r$$, and noting that only one term in the sum uses this variable, we get:

$$\frac{\partial}{\partial \pi_r} \sum_{i=1}^k f(\pi_i) = f'(\pi_r) \quad \quad \quad \text{for any } r = 1,...,k.$$

This reasoning can easily be applied to your situation. Denoting the log-likelihood function by $$\ell_\mathbf{x}$$ and using the chain rule you get:

$$\frac{\partial \ell_\mathbf{x}}{\partial \pi_r} = -c \Big[ \psi (c \pi_r) - \log P_r \Big],$$

where $$\psi$$ is the digamma function. Observe here that differentiating with respect to $$\pi_r$$ removes all terms in the sum that do not have that variable in them, which leaves only one summation term. The overall derivative is just the derivative of this term.

• Thanks a lot! You was very clear! But what about the derivative with respect to rho? In that case there is the summation also in the derivative correct?
– Bibi
Mar 15 at 10:14
• Well, you tell me --- does that varianle occur in each of the terms in the sumation?
– Ben
Mar 15 at 12:14
• Yes, it do not depend on the summation so I need to retain it... thanks! What about the expectation of the digamma below...?
– Bibi
Mar 15 at 14:06
• If you want to ask a new question, you need to post a new question.
– Ben
Mar 15 at 22:07
• Yes, sorry! I have done it. I have asked a new question! stats.stackexchange.com/questions/514014/…
– Bibi
Mar 16 at 11:43