A statistical model for the data set $\bf{y}$ is an exponential family with canonical parameter vector $\theta = (\theta_1,.. \theta_k)$ and canonical statistic $\bf{t(y)} $=$(t_1(\boldsymbol{y}),..t_k(\boldsymbol{y}) ) $ if it has structure

$f(\boldsymbol{y};\theta) = a(\theta)h(\boldsymbol{y})e^{\theta^T\bf{t(y)}} $

under Certain regularity conditions the distribution of $\bf{t}$ has probability function or density function

$f(\boldsymbol{t};\theta) = a(\theta) g(\bf{t})e^{\theta^T \boldsymbol{t}}$

where $g(\boldsymbol{t}) = \sum_{\boldsymbol{t(y)=t}} h(\boldsymbol{y}) $

In the discrete case, the probability function follows immediately by summation over the possible outcomes of $\boldsymbol{y}$ for given $\boldsymbol{t(y) = t}$

Can someone derive this, because I tried but I am not getting there....

$P(t(\boldsymbol{y})=\boldsymbol{t}) = \sum_{t(\boldsymbol{y})=\boldsymbol{t}} P(t(\boldsymbol{y}),\boldsymbol{Y} = \boldsymbol{y}) = \sum_{t(\boldsymbol{y}) = \boldsymbol{t}}P(t(\boldsymbol{y})|\boldsymbol{ Y} = \boldsymbol{y})\cdot a(\theta)h(\boldsymbol{y})e^{\theta^T \boldsymbol{t}} = a(\theta)e^{\theta^T \boldsymbol{t}} \cdot\sum_{t(y) = t} P(t(\boldsymbol{y})|\boldsymbol{Y}=\boldsymbol{y})\cdot h(\boldsymbol{y})$

  • Show us what you tried. This could be a self study question requiring the self study tag. I think canonical statistic is the same as natural parameters. – Michael Chernick May 24 '17 at 16:15
  • This proof in the book is not given, its "up to the reader to check it", I tried what I thought would work.. @MichaelChernick – J.doe May 24 '17 at 17:01
up vote 3 down vote accepted

You're pretty much there. Here are all your steps, with the intended answer added at the end. \begin{align*} P(t(\boldsymbol{y}) = \boldsymbol{t}) &= \sum_{t(\boldsymbol{y}) =\boldsymbol{t}} P(t(\boldsymbol{y})= \mathbf{t}),\boldsymbol{Y} = \boldsymbol{y}) \tag{you} \\ &=\sum_{t(\boldsymbol{y}) = \boldsymbol{t}}P(t(\boldsymbol{y})= \mathbf{t})|\boldsymbol{ Y} = \boldsymbol{y})\cdot a(\theta)h(\boldsymbol{y})e^{\theta^T \boldsymbol{t}} \tag{you}\\ &= a(\theta)e^{\theta^T \boldsymbol{t}} \cdot\sum_{t(y) = t} P(t(\boldsymbol{y}) = \mathbf{t})|\boldsymbol{Y} =\boldsymbol{y})\cdot h(\boldsymbol{y})\tag{you} \\ &= a(\theta) g(\boldsymbol{t}) e^{\theta^T \boldsymbol{t}} \tag{goal} \end{align*} The last line follows because that conditional probability is $1$. If you're conditioning on the fact that $Y=y$, and you're summing over all $y$s such that $t(y) = t$, then that probability must be $1$.

Your Answer


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.