Some questions about exponential families Regarding the book The Bayesian Choice I understand most of chapter three on exponential families,  but there are two parts I have trouble understanding.
The first is

Consider$$f(x|\theta)=h(x)\exp(\theta \cdot x - \psi(\theta))$$a
  generic distribution from an exponential family,
  then a proposition is that a conjugate family for $\theta$ is
  given by
  $$\pi(\theta|\mu , \lambda)=K(\mu,\lambda)\exp(\theta \cdot \mu -
 \lambda \psi(\theta))$$ 
  where $K$ represents the normalising constant and the
  corresponding posterior is $\pi(\theta|\mu+x,\lambda+1)$
  (under certain conditions).

I don't really understand this or how it is applied.
For example consider a $Poisson(\lambda)$ density
$$f(x|\lambda)=\frac{1}{x!}\exp(x\ln\lambda-\lambda)$$
Our sufficient statistic is $x$, and we can write in the form above either as $\psi(\lambda)=\lambda$ or using natural parameter form 
$\psi(n)=\exp(n)$ as our natural parameter is $n=\log \lambda$
But I don't understand the proposition above, how it is applied or used.
Second, I don't understand the following  related proposition from the same page

If $\Theta$ is an open set in $\mathbb{R^{k}}$ and $\theta$ has prior
  $$\pi_{\lambda,x_{0}}(\theta)\propto\exp(\theta \cdot
 x_{0}-\lambda \psi(\theta))$$ with $x_{0}$ in $X$ then $$\text{E}[\nabla \psi
 (\theta)]=\frac{x_{0}}{\lambda}$$ and for $n$ iid observations the conditional posterior
  expectation over all is $\frac{x_{0}+n \bar x}{\lambda+n}$

Looking for any help understanding these or examples of how they are used/why they are important etc.  
 A: Quoting verbatim from my book

It is always possible to reduce an exponential family to a standard
  and minimal form of dimension $m$,
  and this dimension $m$ does not depend on the chosen parameterisation
  (Brown, 1986, pp. 13-16). (See Exercise
  3.20 for an example of a non-regular exponential family.)
Natural exponential families can also be rewritten under the form
  \begin{equation} f(x|\theta) = h(x) e^{\theta.x -\psi(\theta)}
 \tag{3.7} \end{equation} and $\psi(\theta)$ is called the 
  cumulant generating function.

This means that, whatever the original parameterisation of the exponential family, e.g., a Poisson $\mathcal{P}(\lambda)$, there exists a reparameterisation in $\theta$ that looks like the above.
\begin{align*}
f(x|\lambda) &= \frac{\lambda^x}{x!}e^{-\lambda}\\
&= \frac{1}{x!}\exp\{\underbrace{\log(\lambda)}_\text{this is $\theta$}\cdot x-\underbrace{\lambda}_\text{this is $\exp(\theta)$}\}\}\\
&= \frac{1}{x!}\exp\{\theta\cdot x-\underbrace{\exp(\theta)}_\text{this is $\psi(\theta)$}\}\\
&= \underbrace{f^*(x|\theta)}_{\substack{\text{same density}\\ \text{new parameterisation}}}
\end{align*}
This means that the conjugate prior on the natural parameter $\theta\in\mathbb{R}$ is of the form
$$\pi(\theta|x_0,\eta)\propto\exp\{\theta\cdot x_0-\eta\exp(\theta)\}$$
and hence after a change of variable back to $\lambda=\exp(\theta)$, we have that a conjugate prior on the standard parameter $\lambda$ is
$$\pi(\lambda|x_0,\eta)\propto \lambda^{-1} \exp\{\log(\lambda)\cdot x_0-\eta\lambda\}=\lambda^{x_0-1}\,e^{-\eta\lambda}$$
which produces the Gamma $\mathcal{G}a(x_0,\eta)$ distribution as a conjugate prior.
Similarly, for a Binomial $\mathcal{B}in(n,p)$ distribution,
\begin{align*}
f(x|p) &= {n \choose x}\,p^x(1-p)^{n-x}\\
&= {n \choose x}\exp\{\underbrace{\log\{p/(1-p)\}}_\text{this is $\theta$}\cdot x+\underbrace{n\log(1-p)}_\text{this is $-n\log\{1+\exp(\theta)\}$}\}\}\\
&= {n \choose x}\exp\{\theta\cdot x-\underbrace{n\log\{1+\exp(\theta)\}}_\text{this is $\psi(\theta)$}\}\\
&= \underbrace{f^*(x|\theta)}_{\substack{\text{same density}\\ \text{new parameterisation}}}
\end{align*}
Meaning that a conjugate prior on the natural parameter $\theta$ of the Binomial distribution is of the form
$$\pi(\theta|x_0,\lambda) \propto \exp\{\theta\cdot x_0-\underbrace{\lambda\log\{1+\exp(\theta)\}}_\text{$n$ incorporated in $\lambda$}\}$$
For the standard parameter $p\in(0,1)$, this is yet another change of variable from $\theta=\log(p/\{1-p\})$ to $p$:
$$\pi(p|x_0,\lambda) \propto \underbrace{\frac{1}{p(1-p)}}_\text{Jacobian}\,\left(\frac{p}{1-p}\right)^{x_0}\,\exp\{\lambda\log(1-p)\}=p^{x_0-1}\,(1-p)^{\lambda-x_0-1}$$
which returns a Beta $\mathcal{B}e(x_0,\lambda-x_0)$ conjugate distribution (with the constraints $x_0>0$ and $\lambda>x_0$).

Lemma 3.2 If $\theta \in\, \stackrel{\circ }{N}$, the interior set of $N$, the cumulant generating function
  $\psi$ is $\mathcal{C}^{\infty}$ and $$ \mathbb{E}_\theta[x] = \nabla
 \psi(\theta), \qquad \mathrm{cov} (x_i,x_j) =  {\partial^2 \psi\over \partial
 \theta_i \partial \theta_j} (\theta), $$ where $\nabla$ denotes the
  gradient operator.

This is only true for the natural parameterisation (3.7). Which is also used below:

(...) Consider $f(x|\theta) = h(x) e^{\theta\cdot x - \psi(\theta)}$,
  a generic distribution from an exponential family. It then allows for
  a conjugate family, as shown by the following result (whose proof is
  straightforward).
Proposition 3.3 A conjugate family for $f(x|\theta)$ is given by $$
 \pi(\theta|\mu,\lambda)=K(\mu,\lambda)\,e^{\theta\cdot\mu-\lambda\psi(\theta)},
 \tag{3.8} $$ where $K(\mu,\lambda)$ is the normalizing constant of the
  density. The corresponding posterior distribution is
  $\pi(\theta|\mu+x,\lambda+1)$.

This means that, when the prior is the conjugate distribution indexed by the hyper-parameters$$(\mu,\lambda)$$the posterior with one observation is the conjugate distribution indexed by the hyper-parameters$$(\mu+x,\lambda+1)$$and for $n$ observations the conjugate distribution indexed by the hyper-parameters$$(\mu+\sum x_i,\lambda+n)$$This explains for the last part:

Proposition 3.4 If $\Theta$ is an open set in $\mathbb{R}^k$ and $\theta$ has the prior distribution $$ \pi_{\lambda,x_0}(\theta) \propto
 e^{\theta\cdot x_0 -\lambda \psi(\theta)} $$ with $x_0 \in \mathcal{X}$, then
  $$ \mathbb{E}^\pi[\overbrace{\xi(\theta)}^{\substack{\text{notation}\\ \text{for $\mathbb{E}_\theta[X]$}}}]=\mathbb{E}^\pi[\overbrace{\nabla
 \psi(\theta)}^{\substack{\text{as shown in}\\ \text{Lemma 3.2}}}]={x_0\over\lambda}. $$ Therefore, if $x_1,\ldots,x_n$ are
  i.i.d. $f(x|\theta)$, \begin{equation}
 \mathbb{E}^\pi[\xi(\theta)|x_1,\ldots,x_n] = \underbrace{{x_0 + n {\bar x} \over
 \lambda+n}.}_\text{by conjugacy} \tag{3.9} \end{equation}

The proof is done by an integration by part
\begin{align*}\int \nabla \psi(\theta)\,\exp\{-\psi(\theta)+\theta\cdot x_0\}\,\text{d}\theta&=-\int \nabla \exp\{-\psi(\theta)\} \,\exp\{\theta\cdot x_0\}\,\text{d}\theta\\
&=\int \exp\{-\psi(\theta)\}\,\nabla\exp\{\theta\cdot x_0\}\,\text{d}\theta\\
&= \int x_0 \,\exp\{-\psi(\theta)+\theta\cdot x_0\}\,\text{d}\theta
\end{align*} 
Applying this Proposition 3.4 to the Poisson case means (i) getting back to the natural parameterisation as $\theta=\log\{\lambda\}$; (ii) taking the derivative of $\psi(\theta)=\exp\{\theta\}$, i.e. $\psi'(\theta)=\exp\{\theta\}=\lambda=\xi(\theta)=\mathbb{E}_\theta[X]$; (iii) deducing that the posterior expectation of the mean of $X$ is
$$\mathbb{E}^\pi[\lambda|x,x_0,\eta]=\mathbb{E}^\pi[\lambda|x,x_0,\eta]=\frac{x+x_0}{\lambda+1}$$
and
$$\mathbb{E}^\pi[\lambda|x_1,\ldots,x_n,x_0,\eta]=\frac{n\bar{x}_n+x_0}{\lambda+n}$$
Similarly, for the Binomial case, (i) use the natural parameterisation $\theta=\log\{p/(1-p)\}$; (ii) compute the derivative of $\psi(\theta)=\log\{1+e^\theta\}$, i.e., $$\psi'(\theta)=e^\theta/\{1+e^\theta\}=p=\mathbb{E}_p[X]$$
(iii) deduce that the posterior expectation of $p$ under a conjugate prior is
$$\mathbb{E}^\pi[\psi'(\theta)|x,x_0,\lambda]=\mathbb{E}^\pi[p|x,x_0,\lambda]=\underbrace{\frac{x_0+x}{\lambda+n}}_{{\text{remember $n$}\\ \text{integrated in $\lambda$}}}$$
[Surprisingly, this is the second week in a row that I find questions on X validated connected with what I just taught in class!]
