A statistical model for the data set $\bf{y}$ is an exponential family with canonical parameter vector $\theta = (\theta_1,.. \theta_k)$ and canonical statistic $\bf{t(y)} $=$(t_1(\boldsymbol{y}),..t_k(\boldsymbol{y}) ) $ if it has structure

$f(\boldsymbol{y};\theta) = a(\theta)h(\boldsymbol{y})e^{\theta^T\bf{t(y)}} $

under certain regularity conditions the distribution of $\bf{t}$ has probability function or density function

$f(\boldsymbol{t};\theta) = a(\theta) g(\boldsymbol{t})e^{\theta^T \boldsymbol{t}}$

where $g(\boldsymbol{t}) = \int_{\boldsymbol{t(y)=t}} h(\boldsymbol{y})d\boldsymbol{y} \quad (1)$

taking the derivative w.r.t $\theta_j$ of the 'normalising constant': $a^{-1}(\theta) = \int h(\boldsymbol{y})e^{\theta^T \boldsymbol{t}} d\boldsymbol{y}$ gives

$ a^{-1}(\theta)\int t_j(\boldsymbol{y})a(\theta) h(\boldsymbol{y})e^{\theta^T \boldsymbol{t}} d\boldsymbol{y} \quad (2)$

Which according to my book equals $E[t_j] \cdot a^{-1}(\theta)$. But is this correct? because we need $g(\boldsymbol{t})$ instead of $h(\boldsymbol{y})$ in $(2)$ to get the correct expectation.


I do not understand the issue: if $$Y\sim a(\theta)h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\stackrel{\text{def}}{=}h(\boldsymbol{y})e^{\theta^T\bf{t(y)}-\log\Psi(\theta)}$$then (Brown, 1986)$$\mathbb{E}_\theta[\bf{t(Y)}]=\nabla \Psi(\theta)$$ where $\nabla$ denontes the gradient in $\theta$, that is the vector of derivatives against the components of $\theta$. By construction$$a^{-1}(\theta)=\int h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\,\text{d}\bf{y}$$and thus \begin{align} \nabla \Psi(\theta) &= \nabla \log a^{-1}(\theta)\\ &= \nabla a^{-1}(\theta) \big/ a^{-1}(\theta)\\ &= a(\theta)\, \nabla \int h(\boldsymbol{y})e^{\theta^T\bf{t(y)}}\,\text{d}{\bf{y}}\tag{switch derivative}\\ &= a(\theta)\, \int h({\boldsymbol y})\nabla \left\{e^{\theta^T{\bf{t(y)}}}\right\}\,\text{d}{\bf{y}}\tag{and integral}\\ &= a(\theta)\, \int h(\boldsymbol{y})\left\{{\bf{t(y)}} e^{\theta^T{\bf{t(y)}}}\right\}\,\text{d}{\bf{y}}\\ &= a(\theta)\, \int {\bf t(y)} h({\bf{y}})e^{\theta^T{\bf t(y)}}h(\boldsymbol{y})\,\text{d}\bf{y} \\ &= \mathbb{E}_\theta[{\bf t(Y)}]\tag{${\bf T}={\bf t(Y)}$}\\ &= \mathbb{E}_\theta[{\bf T}]\\ &= a(\theta)\, \int {\bf{t}} e^{\theta^T{\bf{t}}}\,g({\bf t})\,\text{d}{\bf{t}}\tag{density of $\bf T$} \end{align} meaning that both expressions are correct $$\int {\bf t(y)} e^{\theta^T{\bf t(y)}}h({\bf{y}})\,\text{d}{\bf{y}}=\int {\bf{t}} e^{\theta^T{\bf{t}}}\,g({\bf t})\,\text{d}\bf{t}$$ Note that as pointed out by W. Huber the representation (1) $$\int_{\boldsymbol{t(y)=t}} h(\boldsymbol{y})\text{d}\boldsymbol{y}$$ is incorrect (or meaningless) outside the discrete case.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.