How to derive the expectation of $\ln \mu_j$ in Dirichlet distribution I have derived the mean and variance of $\mu_j$ in Dirichlet distribution $\text{Dir}(\mu_1, \cdots, \mu_K|\alpha_1, \cdots, \alpha_k)$.
On https://en.wikipedia.org/wiki/Dirichlet_distribution, it also shows that
$$\mathbb{E}\left[\ln [\mu_j]\right] = \psi(\alpha_j) - \psi(\alpha_0)$$
where

*

*$\alpha_0 = \sum_{k=1}^K \alpha_k$, and

*$\psi(\alpha) = \frac{d}{d \alpha} \ln \Gamma(\alpha)$, the digamma function.

Can anyone provide hints or suggestion on how $\mathbb{E}\left[\ln \mu_j\right]$ can be derived, please?
 A: \begin{align}
\mathbb{E}[\ln \mu_j]
&= \int_0^1 \ln \mu_j \text{Dir}(\boldsymbol{\mu}|\boldsymbol{\alpha}) d\mu_j \\
&= \int_0^1 \ln \mu_j \text{Beta}(\alpha_j, \alpha_0 - \alpha_j) d\mu_j \\
&= \int_0^1 \ln \mu_j \frac{1}{\text{B}(\alpha_j, \alpha_0 - \alpha_j)}  \mu_j^{\alpha_j - 1} (1 - \mu_j)^{\alpha_0 - \alpha_j - 1} d\mu_j \\
&= \frac{1}{\text{B}(\alpha_j, \alpha_0 - \alpha_j)}  \int_0^1 \frac{d \mu_j^{\alpha_j - 1}}{d \alpha_j}(1 - \mu_j)^{\alpha_0 - \alpha_j - 1} d\mu_j \\
&= \frac{1}{\text{B}(\alpha_j, \alpha_0 - \alpha_j)}  \frac{d}{d \alpha_j} \int_0^1  \mu_j^{\alpha_j - 1} (1 - \mu_j)^{\alpha_0 - \alpha_j - 1} d\mu_j \\
&= \frac{1}{\text{B}(\alpha_j, \alpha_0 - \alpha_j)}  \frac{d \text{B}(\alpha_j, \alpha_0 - \alpha_j)}{d \alpha_j} \\
&= \frac{d}{\alpha_j} \ln \text{B}(\alpha_j, \alpha_0 - \alpha_j) \\
&= \frac{d}{\alpha_j} \ln \frac{\Gamma(\alpha_j) \Gamma(\alpha_0 - \alpha_j)} {\Gamma(\alpha_0)}  \\
&= \frac{d}{\alpha_j} \bigg( \ln \Gamma(\alpha_j) + \ln \Gamma(\alpha_0 - \alpha_j) - \ln \Gamma(\alpha_0) \bigg ) \\
&= \frac{d}{\alpha_j} \ln \Gamma(\alpha_j) - \frac{d}{\alpha_j}\ln \Gamma(\alpha_0) \\
&= \frac{d}{\alpha_j} \ln \Gamma(\alpha_j) - \frac{d}{\alpha_0}\ln \Gamma(\alpha_0) \\
&= \psi(\alpha_j) - \psi(\alpha_0)
\end{align}
Note:

*

*In the 4th equality, we used the fact $\frac{d}{dx} a^x = a^x \ln a$ as shown below.

*In the 4th last and 2nd last equalities, when taking derivative wrt. $\alpha_j$, $\alpha_0 - \alpha_j$ is considered a constant, $\alpha_0$ is NOT a constant, so

*

*$\frac{d}{\alpha_j} \ln(\alpha_0 - \alpha_j) = 0$ (4th last equality), and

*$\frac{d}{\alpha_j}\ln \Gamma(\alpha_0) = \frac{d}{\alpha_j + (\alpha_0 - \alpha_j)}\ln \Gamma(\alpha_0) = \frac{d}{\alpha_0}\ln \Gamma(\alpha_0)$ (2nd last equality).



*$\psi(x) \equiv \frac{d}{dx} \ln \Gamma(x) $ is called the digamma function.

A: The answer from @zxyue is fantastic.
However, it omitted the process how the 1st equation $\mathbb{E}[\ln\mu_j]=\int_0^1 \ln \mu_j \text{Dir}(\boldsymbol{\mu}|\boldsymbol{\alpha}) d\mu_j$ is achieved. This answer is an supplementary answer to @zxyue's answer.
According to the definition of Expectation: $\mathbb{E}[X]=\int_\mathbb{R}xf(x)dx$. Then
$$
\begin{align}
\mathbb{E}[\ln\mu_j]
&=\int\ln\mu_jf(\boldsymbol{\mu})d\boldsymbol{\mu}\\
&=\int\dots\int\ln\mu_jf(\boldsymbol{\mu})d\mu_1\dots d\mu_K\\
&=\int\dots\int\int\int\dots\int\ln\mu_jf(\boldsymbol{\mu})d\mu_1\dots d\mu_{j-1}d\mu_jd\mu_{j+1}\dots d\mu_K
\end{align}
$$
Move integral $\int\dots d\mu_j$ to the most outside and set its interval as [0,1]
$$
\mathbb{E}[\ln\mu_j]
=\int_0^1\int_0^{1-\sum_{k=1}^{\mathbb{K}-1}{\mu_k}}\dots\int_0^{1-\sum_{k=1}^{j}{\mu_k}}\int_0^{1-\mu_j-\sum_{k=1}^{j-2}\mu_k}\dots\int_0^{1-\mu_j}\ln\mu_jf(\boldsymbol{\mu})d\mu_1\dots d\mu_{j-1}d\mu_{j+1}\dots d\mu_Kd\mu_j
$$
Move $ln\mu_j$ out of the core integral then to the outermost integral, since $ln\mu_j$ is independent of $\mu_1\dots\mu_{j-1},\mu_{j+1}\dots\mu_K$ thus could be treated as a constant
$$
\begin{align}
\mathbb{E}[\ln\mu_j]
&=\int_0^1\ln\mu_j\int_0^{1-\sum_{k=1}^{\mathbb{K}-1}{\mu_k}}\dots\int_0^{1-\sum_{k=1}^{j}{\mu_k}}\int_0^{1-\mu_j-\sum_{k=1}^{j-2}\mu_k}\dots\int_0^{1-\mu_j}f(\boldsymbol{\mu})d\mu_1\dots d\mu_{j-1}d\mu_{j+1}\dots d\mu_Kd\mu_j\\
&=\int_0^1\ln\mu_jg(\boldsymbol{\mu})d\mu_j
\end{align}
$$
Where $g(\boldsymbol{\mu})$ is $\int_0^{1-\sum_{k=1}^{\mathbb{K}-1}{\mu_k}}\dots\int_0^{1-\sum_{k=1}^{j}{\mu_k}}\int_0^{1-\mu_j-\sum_{k=1}^{j-2}\mu_k}\dots\int_0^{1-\mu_j}f(\boldsymbol{\mu})d\mu_1\dots d\mu_{j-1}d\mu_{j+1}\dots d\mu_K$
By observation, we could easily realize that $g(\boldsymbol{\mu})$ is the marginal distribution of variable ${M_j}$. That is to say
$$
\begin{align}
g(\boldsymbol{\mu})&=\int_0^{1-\sum_{k=1}^{\mathbb{K}-1}{\mu_k}}\dots\int_0^{1-\sum_{k=1}^{j}{\mu_k}}\int_0^{1-\mu_j-\sum_{k=1}^{j-2}\mu_k}\dots\int_0^{1-\mu_j}f(\boldsymbol{\mu})d\mu_1\dots d\mu_{j-1}d\mu_{j+1}\dots d\mu_K\\
&=f_{M_j}(\mu_j)
\end{align}
$$
As we known, $\boldsymbol{\mu}$ follows Dirichlet Distribution $\text{Dir}(\boldsymbol{\mu}|\boldsymbol{\alpha})$. Dirichlet Distribution is a multivariate version of Beta Distribution. Intuitively, we could get that $f_{M_j}(\mu_j)$ is the probability density function of Beta Distribution $\text{Beta}(\alpha_j, \alpha_0 - \alpha_j)$, where $\alpha_0=\sum_{j=1}^{K}\alpha_j$. Then we could write $\mathbb{E}[\ln\mu_j]$ as follows
$$
\begin{align}
\mathbb{E}[\ln\mu_j]
&=\int\ln\mu_jf(\boldsymbol{\mu})d\boldsymbol{\mu}\\
&= \int_0^1\ln\mu_jg(\boldsymbol{\mu})d\mu_j\\
&= \int_0^1\ln\mu_jf_{M_j}(\mu_j)d\mu_j\\
&= \int_0^1 \ln \mu_j \text{Beta}(\alpha_j, \alpha_0 - \alpha_j) d\mu_j
\end{align}
$$
Which is now consistent with the 2nd equation from @zxyue's answer.
For a formal derivation of the marginal distribution of Dirichlet distribution, please refer the answer from question Find marginal distribution of -variate Dirichlet
