3
$\begingroup$

I need to compute the first derivatives of the Dirichlet distribution, defined in the following way:

$$r(P; \pi, \rho) = \frac{\Gamma(c)}{\prod_{i=1}^{k} \Gamma(c \pi_i)} \cdot \prod_{i=1}^{k} P_i^{c\pi_i - 1},$$

where $c=\rho^{-2}(1-\rho^2) = \sum_{i=1}^{k} \alpha_i$. Now I need to compute the first derivative of the log likelihood with respect to $\pi_i$ and $\rho$ but I am finding myself in having a hard time. I defined the log likelihood as: $$\log\Gamma(c) - \sum_{i=1}^{k} \log\Gamma(c\pi_i) + \sum_{i=1}^{k}(c\pi_i-1) \log P_i,$$ and I hope it is correct. But then, I don't get how I can compute the derivatives, in particular the one with respect to $\pi_i$ because I have the summation with respect to i and a digamma function. Can somebody show it to me? just the derivative with respect to $\pi_i$ of the second and third term of the log likelihood.

And also: when I have the log of a gamma function I know that the derivative is the digamma function. But do I have to multiply the digamma function for the derivative of the argument of the digamma function (like chain rule of the derivatives)?

Thanks in advance!


Ok, thanks a lot! I still have a doubt about the digamma and trigamma function. I need to compute the Fisher information and so after having computed the first derivative I compute also the second derivative which is actually equal to $$-c^2 * \psi'(c\pi_i)$$.

I have to compute the $E_P[-d^2l/d\pi_i^2]$ and I am given the result of this which is equal to $c^2[\psi'(c\pi_i) + \psi'(c\pi_k)]$ for $i = 1,...,k-1$. But how can I get this result? I don't get why this is the result... from where they took $\psi'(c\pi_k)$?

And also which is the expectation of a digamma and trigamma function? Thanks a lot again if you can help me!

$\endgroup$
2
  • $\begingroup$ Is $\pi$ a vector with component $\pi_i$? $\endgroup$
    – microhaus
    Mar 14 at 12:53
  • $\begingroup$ Yes exactly, with k elements $\endgroup$
    – Bibi
    Mar 14 at 13:40
5
$\begingroup$

You are confusing yourself here by failing to recognise that the index of summation $i$ is just an index, not a variable in the equation. Consequently, if you expand out the sum, you get a form that does not have $i$ in it:

$$\sum_{i=1}^k f(\pi_i) = f(\pi_1) + \cdots + f(\pi_k).$$

To avoid confusion, it is good practice not to use the same index for your derivative and for your summation index. Taking the partial derivative with respect to an arbitrary element $\pi_r$, and noting that only one term in the sum uses this variable, we get:

$$\frac{\partial}{\partial \pi_r} \sum_{i=1}^k f(\pi_i) = f'(\pi_r) \quad \quad \quad \text{for any } r = 1,...,k.$$

This reasoning can easily be applied to your situation. Denoting the log-likelihood function by $\ell_\mathbf{x}$ and using the chain rule you get:

$$\frac{\partial \ell_\mathbf{x}}{\partial \pi_r} = -c \Big[ \psi (c \pi_r) - \log P_r \Big],$$

where $\psi$ is the digamma function. Observe here that differentiating with respect to $\pi_r$ removes all terms in the sum that do not have that variable in them, which leaves only one summation term. The overall derivative is just the derivative of this term.

$\endgroup$
5
  • $\begingroup$ Thanks a lot! You was very clear! But what about the derivative with respect to rho? In that case there is the summation also in the derivative correct? $\endgroup$
    – Bibi
    Mar 15 at 10:14
  • $\begingroup$ Well, you tell me --- does that varianle occur in each of the terms in the sumation? $\endgroup$
    – Ben
    Mar 15 at 12:14
  • $\begingroup$ Yes, it do not depend on the summation so I need to retain it... thanks! What about the expectation of the digamma below...? $\endgroup$
    – Bibi
    Mar 15 at 14:06
  • $\begingroup$ If you want to ask a new question, you need to post a new question. $\endgroup$
    – Ben
    Mar 15 at 22:07
  • $\begingroup$ Yes, sorry! I have done it. I have asked a new question! stats.stackexchange.com/questions/514014/… $\endgroup$
    – Bibi
    Mar 16 at 11:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.