# property of the normalising constant in an exponential family

A statistical model for the data set $\bf{y}$ is an exponential family with canonical parameter vector $\theta = (\theta_1,.. \theta_k)$ and canonical statistic $\bf{t(y)}$=$(t_1(\boldsymbol{y}),..t_k(\boldsymbol{y}) )$ if it has structure

$f(\boldsymbol{y};\theta) = a(\theta)h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}$

under certain regularity conditions the distribution of $\bf{t}$ has probability function or density function

$f(\boldsymbol{t};\theta) = a(\theta) g(\boldsymbol{t})e^{\theta^T \boldsymbol{t}}$

where $g(\boldsymbol{t}) = \int_{\boldsymbol{t(y)=t}} h(\boldsymbol{y})d\boldsymbol{y} \quad (1)$

taking the derivative w.r.t $\theta_j$ of the 'normalising constant': $a^{-1}(\theta) = \int h(\boldsymbol{y})e^{\theta^T \boldsymbol{t}} d\boldsymbol{y}$ gives

$a^{-1}(\theta)\int t_j(\boldsymbol{y})a(\theta) h(\boldsymbol{y})e^{\theta^T \boldsymbol{t}} d\boldsymbol{y} \quad (2)$

Which according to my book equals $E[t_j] \cdot a^{-1}(\theta)$. But is this correct? because we need $g(\boldsymbol{t})$ instead of $h(\boldsymbol{y})$ in $(2)$ to get the correct expectation.

I do not understand the issue: if $$Y\sim a(\theta)h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\stackrel{\text{def}}{=}h(\boldsymbol{y})e^{\theta^T\bf{t(y)}-\log\Psi(\theta)}$$then (Brown, 1986)$$\mathbb{E}_\theta[\bf{t(Y)}]=\nabla \Psi(\theta)$$ where $\nabla$ denontes the gradient in $\theta$, that is the vector of derivatives against the components of $\theta$. By construction$$a^{-1}(\theta)=\int h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\,\text{d}\bf{y}$$and thus \begin{align} \nabla \Psi(\theta) &= \nabla \log a^{-1}(\theta)\\ &= \nabla a^{-1}(\theta) \big/ a^{-1}(\theta)\\ &= a(\theta)\, \nabla \int h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\,\text{d}{\bf{y}}\tag{switch derivative}\\ &= a(\theta)\, \int h({\boldsymbol y})\nabla \left\{e^{\theta^T{\bf{t(y)}}}\right\}\,\text{d}{\bf{y}}\tag{and integral}\\ &= a(\theta)\, \int h(\boldsymbol{y})\left\{{\bf{t(y)}} e^{\theta^T{\bf{t(y)}}}\right\}\,\text{d}{\bf{y}}\\ &= a(\theta)\, \int {\bf t(y)} h({\bf{y}})e^{\theta^T{\bf t(y)}}h(\boldsymbol{y})\,\text{d}\bf{y} \\ &= \mathbb{E}_\theta[{\bf t(Y)}]\tag{${\bf T}={\bf t(Y)}$}\\ &= \mathbb{E}_\theta[{\bf T}]\\ &= a(\theta)\, \int {\bf{t}} e^{\theta^T{\bf{t}}}\,g({\bf t})\,\text{d}{\bf{t}}\tag{density of $\bf T$} \end{align} meaning that both expressions are correct $$\int {\bf t(y)} e^{\theta^T{\bf t(y)}}h({\bf{y}})\,\text{d}{\bf{y}}=\int {\bf{t}} e^{\theta^T{\bf{t}}}\,g({\bf t})\,\text{d}\bf{t}$$ Note that as pointed out by W. Huber the representation (1) $$\int_{\boldsymbol{t(y)=t}} h(\boldsymbol{y})\text{d}\boldsymbol{y}$$ is incorrect (or meaningless) outside the discrete case.