A statistical model for the data set $\bf{y}$ is an exponential family with canonical parameter vector $\theta = (\theta_1,.. \theta_k)$ and canonical statistic $\bf{t(y)} $=$(t_1(\boldsymbol{y}),..t_k(\boldsymbol{y}) ) $ if it has structure

$f(\boldsymbol{y};\theta) = a(\theta)h(\boldsymbol{y})e^{\theta^T\bf{t(y)}} $

under certain regularity conditions the distribution of $\bf{t}$ has probability function or density function

$f(\boldsymbol{t};\theta) = a(\theta) g(\boldsymbol{t})e^{\theta^T \boldsymbol{t}}$

where $g(\boldsymbol{t}) = \int_{\boldsymbol{t(y)=t}} h(\boldsymbol{y})d\boldsymbol{y} \quad (1)$

taking the derivative w.r.t $\theta_j$ of the 'normalising constant': $a^{-1}(\theta) = \int h(\boldsymbol{y})e^{\theta^T \boldsymbol{t}} d\boldsymbol{y}$ gives

$ a^{-1}(\theta)\int t_j(\boldsymbol{y})a(\theta) h(\boldsymbol{y})e^{\theta^T \boldsymbol{t}} d\boldsymbol{y} \quad (2)$

Which according to my book equals $E[t_j] \cdot a^{-1}(\theta)$. But is this correct? because we need $g(\boldsymbol{t})$ instead of $h(\boldsymbol{y})$ in $(2)$ to get the correct expectation.


1 Answer 1


I do not understand the issue: if $$Y\sim a(\theta)h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\stackrel{\text{def}}{=}h(\boldsymbol{y})e^{\theta^T\bf{t(y)}-\log\Psi(\theta)}$$then (Brown, 1986)$$\mathbb{E}_\theta[\bf{t(Y)}]=\nabla \Psi(\theta)$$ where $\nabla$ denontes the gradient in $\theta$, that is the vector of derivatives against the components of $\theta$. By construction$$a^{-1}(\theta)=\int h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\,\text{d}\bf{y}$$and thus \begin{align} \nabla \Psi(\theta) &= \nabla \log a^{-1}(\theta)\\ &= \nabla a^{-1}(\theta) \big/ a^{-1}(\theta)\\ &= a(\theta)\, \nabla \int h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\,\text{d}{\bf{y}}\tag{switch derivative}\\ &= a(\theta)\, \int h({\boldsymbol y})\nabla \left\{e^{\theta^T{\bf{t(y)}}}\right\}\,\text{d}{\bf{y}}\tag{and integral}\\ &= a(\theta)\, \int h(\boldsymbol{y})\left\{{\bf{t(y)}} e^{\theta^T{\bf{t(y)}}}\right\}\,\text{d}{\bf{y}}\\ &= a(\theta)\, \int {\bf t(y)} h({\bf{y}})e^{\theta^T{\bf t(y)}}h(\boldsymbol{y})\,\text{d}\bf{y} \\ &= \mathbb{E}_\theta[{\bf t(Y)}]\tag{${\bf T}={\bf t(Y)}$}\\ &= \mathbb{E}_\theta[{\bf T}]\\ &= a(\theta)\, \int {\bf{t}} e^{\theta^T{\bf{t}}}\,g({\bf t})\,\text{d}{\bf{t}}\tag{density of $\bf T$} \end{align} meaning that both expressions are correct $$\int {\bf t(y)} e^{\theta^T{\bf t(y)}}h({\bf{y}})\,\text{d}{\bf{y}}=\int {\bf{t}} e^{\theta^T{\bf{t}}}\,g({\bf t})\,\text{d}\bf{t}$$ Note that as pointed out by W. Huber the representation (1) $$\int_{\boldsymbol{t(y)=t}} h(\boldsymbol{y})\text{d}\boldsymbol{y}$$ is incorrect (or meaningless) outside the discrete case.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.