Quoting verbatim from my book
It is always possible to reduce an exponential family to a standard
and minimal form of dimension $m$,
and this dimension $m$ does not depend on the chosen parameterisation
(Brown, 1986, pp. 13-16). (See Exercise
3.20 for an example of a non-regular exponential family.)
Natural exponential families can also be rewritten under the form
\begin{equation} f(x|\theta) = h(x) e^{\theta.x -\psi(\theta)}
\tag{3.7} \end{equation} and $\psi(\theta)$ is called the
cumulant generating function.
This means that, whatever the original parameterisation of the exponential family, e.g., a Poisson $\mathcal{P}(\lambda)$, there exists a reparameterisation in $\theta$ that looks like the above.
\begin{align*}
f(x|\lambda) &= \frac{\lambda^x}{x!}e^{-\lambda}\\
&= \frac{1}{x!}\exp\{\underbrace{\log(\lambda)}_\text{this is $\theta$}\cdot x-\underbrace{\lambda}_\text{this is $\exp(\theta)$}\}\}\\
&= \frac{1}{x!}\exp\{\theta\cdot x-\underbrace{\exp(\theta)}_\text{this is $\psi(\theta)$}\}\\
&= \underbrace{f^*(x|\theta)}_{\substack{\text{same density}\\ \text{new parameterisation}}}
\end{align*}
This means that the conjugate prior on the natural parameter $\theta\in\mathbb{R}$ is of the form
$$\pi(\theta|x_0,\eta)\propto\exp\{\theta\cdot x_0-\eta\exp(\theta)\}$$
and hence after a change of variable back to $\lambda=\exp(\theta)$, we have that a conjugate prior on the standard parameter $\lambda$ is
$$\pi(\lambda|x_0,\eta)\propto \lambda^{-1} \exp\{\log(\lambda)\cdot x_0-\eta\lambda\}=\lambda^{x_0-1}\,e^{-\eta\lambda}$$
which produces the Gamma $\mathcal{G}a(x_0,\eta)$ distribution as a conjugate prior.
Similarly, for a Binomial $\mathcal{B}in(n,p)$ distribution,
\begin{align*}
f(x|p) &= {n \choose x}\,p^x(1-p)^{n-x}\\
&= {n \choose x}\exp\{\underbrace{\log\{p/(1-p)\}}_\text{this is $\theta$}\cdot x+\underbrace{n\log(1-p)}_\text{this is $-n\log\{1+\exp(\theta)\}$}\}\}\\
&= {n \choose x}\exp\{\theta\cdot x-\underbrace{n\log\{1+\exp(\theta)\}}_\text{this is $\psi(\theta)$}\}\\
&= \underbrace{f^*(x|\theta)}_{\substack{\text{same density}\\ \text{new parameterisation}}}
\end{align*}
Meaning that a conjugate prior on the natural parameter $\theta$ of the Binomial distribution is of the form
$$\pi(\theta|x_0,\lambda) \propto \exp\{\theta\cdot x_0-\underbrace{\lambda\log\{1+\exp(\theta)\}}_\text{$n$ incorporated in $\lambda$}\}$$
For the standard parameter $p\in(0,1)$, this is yet another change of variable from $\theta=\log(p/\{1-p\})$ to $p$:
$$\pi(p|x_0,\lambda) \propto \underbrace{\frac{1}{p(1-p)}}_\text{Jacobian}\,\left(\frac{p}{1-p}\right)^{x_0}\,\exp\{\lambda\log(1-p)\}=p^{x_0-1}\,(1-p)^{\lambda-x_0-1}$$
which returns a Beta $\mathcal{B}e(x_0,\lambda-x_0)$ conjugate distribution (with the constraints $x_0>0$ and $\lambda>x_0$).
Lemma 3.2 If $\theta \in\, \stackrel{\circ }{N}$, the interior set of $N$, the cumulant generating function
$\psi$ is $\mathcal{C}^{\infty}$ and $$ \mathbb{E}_\theta[x] = \nabla
\psi(\theta), \qquad \mathrm{cov} (x_i,x_j) = {\partial^2 \psi\over \partial
\theta_i \partial \theta_j} (\theta), $$ where $\nabla$ denotes the
gradient operator.
This is only true for the natural parameterisation (3.7). Which is also used below:
(...) Consider $f(x|\theta) = h(x) e^{\theta\cdot x - \psi(\theta)}$,
a generic distribution from an exponential family. It then allows for
a conjugate family, as shown by the following result (whose proof is
straightforward).
Proposition 3.3 A conjugate family for $f(x|\theta)$ is given by $$
\pi(\theta|\mu,\lambda)=K(\mu,\lambda)\,e^{\theta\cdot\mu-\lambda\psi(\theta)},
\tag{3.8} $$ where $K(\mu,\lambda)$ is the normalizing constant of the
density. The corresponding posterior distribution is
$\pi(\theta|\mu+x,\lambda+1)$.
This means that, when the prior is the conjugate distribution indexed by the hyper-parameters$$(\mu,\lambda)$$the posterior with one observation is the conjugate distribution indexed by the hyper-parameters$$(\mu+x,\lambda+1)$$and for $n$ observations the conjugate distribution indexed by the hyper-parameters$$(\mu+\sum x_i,\lambda+n)$$This explains for the last part:
Proposition 3.4 If $\Theta$ is an open set in $\mathbb{R}^k$ and $\theta$ has the prior distribution $$ \pi_{\lambda,x_0}(\theta) \propto
e^{\theta\cdot x_0 -\lambda \psi(\theta)} $$ with $x_0 \in \mathcal{X}$, then
$$ \mathbb{E}^\pi[\overbrace{\xi(\theta)}^{\substack{\text{notation}\\ \text{for $\mathbb{E}_\theta[X]$}}}]=\mathbb{E}^\pi[\overbrace{\nabla
\psi(\theta)}^{\substack{\text{as shown in}\\ \text{Lemma 3.2}}}]={x_0\over\lambda}. $$ Therefore, if $x_1,\ldots,x_n$ are
i.i.d. $f(x|\theta)$, \begin{equation}
\mathbb{E}^\pi[\xi(\theta)|x_1,\ldots,x_n] = \underbrace{{x_0 + n {\bar x} \over
\lambda+n}.}_\text{by conjugacy} \tag{3.9} \end{equation}
The proof is done by an integration by part
\begin{align*}\int \nabla \psi(\theta)\,\exp\{-\psi(\theta)+\theta\cdot x_0\}\,\text{d}\theta&=-\int \nabla \exp\{-\psi(\theta)\} \,\exp\{\theta\cdot x_0\}\,\text{d}\theta\\
&=\int \exp\{-\psi(\theta)\}\,\nabla\exp\{\theta\cdot x_0\}\,\text{d}\theta\\
&= \int x_0 \,\exp\{-\psi(\theta)+\theta\cdot x_0\}\,\text{d}\theta
\end{align*}
Applying this Proposition 3.4 to the Poisson case means (i) getting back to the natural parameterisation as $\theta=\log\{\lambda\}$; (ii) taking the derivative of $\psi(\theta)=\exp\{\theta\}$, i.e. $\psi'(\theta)=\exp\{\theta\}=\lambda=\xi(\theta)=\mathbb{E}_\theta[X]$; (iii) deducing that the posterior expectation of the mean of $X$ is
$$\mathbb{E}^\pi[\lambda|x,x_0,\eta]=\mathbb{E}^\pi[\lambda|x,x_0,\eta]=\frac{x+x_0}{\lambda+1}$$
and
$$\mathbb{E}^\pi[\lambda|x_1,\ldots,x_n,x_0,\eta]=\frac{n\bar{x}_n+x_0}{\lambda+n}$$
Similarly, for the Binomial case, (i) use the natural parameterisation $\theta=\log\{p/(1-p)\}$; (ii) compute the derivative of $\psi(\theta)=\log\{1+e^\theta\}$, i.e., $$\psi'(\theta)=e^\theta/\{1+e^\theta\}=p=\mathbb{E}_p[X]$$
(iii) deduce that the posterior expectation of $p$ under a conjugate prior is
$$\mathbb{E}^\pi[\psi'(\theta)|x,x_0,\lambda]=\mathbb{E}^\pi[p|x,x_0,\lambda]=\underbrace{\frac{x_0+x}{\lambda+n}}_{{\text{remember $n$}\\ \text{integrated in $\lambda$}}}$$
[Surprisingly, this is the second week in a row that I find questions on X validated connected with what I just taught in class!]
