I am trying to find the Moments of the natural statistics of the normal-gamma distribution. $$(X,T) - NormalGamma(\mu, \lambda,\alpha,\beta)$$ I found on its Wikipedia page that the moments of the natural statistics are: $$ \begin{align} E(\ln T) &= \psi(\alpha) - \ln \beta,\\ E(T) &= \frac{\alpha}{\beta},\\ E(TX) &= \mu \frac{\alpha}{\beta},\\ E(T^2X) &= \frac{1}{\lambda} + \mu^2 \frac{\alpha}{\beta}. \end{align} $$ The density function is $$ f(x,t;\mu,\lambda,\alpha,\beta) = \frac{\beta^\alpha \sqrt{\lambda}}{\Gamma(\alpha)\sqrt{2\pi}} t^{\alpha-\frac{1}{2}} e^{-\beta t} \exp\left(-\frac{\lambda t(x-\mu)^2}{2} \right). $$ How are the above expectations derived?

  • 1
    $\begingroup$ A crucial starting point is to define the pdf of the distribution you are interested in, given that there are usually multiple distributions sharing the same name, and to define $T$ and $X$ and the parameters. Absent that, your question does not have an answer. $\endgroup$
    – wolfies
    Commented Mar 14, 2022 at 21:33
  • $\begingroup$ The pdf was already there, for T and X I thought they were clear for someone who knows this distribution $\endgroup$
    – sam
    Commented Mar 15, 2022 at 14:15
  • $\begingroup$ As the Wikipedia article states, these moments are readily derived from the moment generating function of $(X,T).$ $\endgroup$
    – whuber
    Commented Mar 15, 2022 at 14:23
  • $\begingroup$ I know but I need the proof on how that was done $\endgroup$
    – sam
    Commented Mar 15, 2022 at 14:28

1 Answer 1


First, note that normal-gamma distribution is an exponential family and has the following form: $$ f(x;\eta) = h(x)\exp\{\color{blue}{\eta^\top t(x)} \color{red}{-A(\eta)} \}, $$ where $\eta$ is the natural parameter, $t(x)$ is the sufficient statistic, $h(x)$ is the underlying measure, and $A(\eta)$ is the log-normalizing constant. What's interesting about the log-normalizing constant $A(\eta)$ is that its derivatives provide the moments of the sufficient statistic, i.e., $$ \begin{align} \frac{d}{d\eta}A(\eta) &= \dfrac{d}{d\eta}\left(\log \int \exp\{\eta^\top t(x)\} h(x)\,dx \right)\\ &= \frac{\int t(x)\exp\{\eta^\top t(x)\}h(x)\,dx}{\int \exp\{\eta^\top t(x)\}h(x)\,dx}\\ &= \int t(x) \exp\{\eta^\top t(x)-A(\eta)\}h(x)\,dx\\ &= E(t(X)). \end{align} $$

Let's rewrite the density in terms of its natural parameter and sufficient statistic. Observe that $$ \begin{align} f(X,T;\mu,\lambda,\alpha,\beta) &= \exp\Bigg\{\color{blue}{\left(\alpha-\frac{1}{2} \right)\log T -\left(\beta + \frac{\lambda \mu^2}{2} \right)T + \lambda \mu XT - \frac{\lambda}{2} TX^2} \\ &\quad \quad\color{red}{+\alpha \log \beta + \frac{1}{2}\log \lambda - \log \Gamma(\alpha) - \frac{1}{2}\log (2\pi)}\Bigg\}. \end{align} $$ This part is merely matching the shape of the general form of an exponential family and a specific example. I colored the parts so each component is clearly separated from one another. Now, it's pretty clear from the blue part that $$ \eta_1 = \alpha - \frac{1}{2},\;\eta_2 = -\beta - \frac{\lambda\mu^2}{2},\;\eta_3=\lambda\mu,\;\eta_4=-\frac{\lambda}{2}, $$ and $$ T_1 = \ln T,\;T_2=T,\; T_3=TX,\;T_4=TX^2, $$ so that $\eta^\top t(x) = \eta_1T_1 + \eta_2T_2 + \eta_3T_3 + \eta_4T_4$.

Next step is to reexpress $A(\eta)$ in terms of $\eta$. Note that $\alpha= \eta_1+\frac{1}{2}$, $\beta=-\eta_2 + \frac{\eta_3^2}{4\eta_4}$, $\mu = -\frac{\eta_3}{2\eta_4}$, and $\lambda = -2\eta_4$. Then, $$ \begin{align} A(\eta)&= -\left(\eta_1+\dfrac{1}{2} \right)\log\left(\frac{\eta_3^2}{4\eta_4} - \eta_2 \right) -\frac{1}{2}\log(-2\eta_4) +\log\Gamma\left(\eta_1+\frac{1}{2}\right) + \frac{1}{2}\log(2\pi). \end{align} $$

From now on, it's good ol' differentiation: $$ \begin{align} \frac{d}{d\eta_1}A(\eta) &= -\log\left(\frac{\eta_3^2}{4\eta_4} - \eta_2 \right)+ \psi\left( \eta_1+\frac{1}{2}\right) = \psi(\alpha)-\log(\beta),\\ \frac{d}{d\eta_2}A(\eta) &= \frac{\eta_1+\frac{1}{2}}{\frac{\eta_2^2}{4\eta_4}-\eta_2} = \frac{\alpha}{\beta},\\ \frac{d}{d\eta_3}A(\eta) &= -\left(\eta_1+\frac{1}{2} \right)\frac{\eta_3/(2\eta_4)}{\frac{\eta_3^2}{4\eta_4} - \eta_2} = \frac{\alpha \mu}{\beta},\\ \frac{d}{d\eta_4}A(\eta) &= -\left(\eta_1+\frac{1}{2} \right)\frac{-\frac{\eta_3^2}{4\eta_4^2}}{\frac{\eta_3^2}{4\eta_4} - \eta_2} - \frac{1}{2\eta_4} = \frac{\alpha\mu^2}{\beta} + \frac{1}{\lambda}. \end{align} $$

  • $\begingroup$ Thank you for your help, but there are some points that I don't understand. The first one is how you managed to write the normal gamma in the exponential family format, you said it's merely matching but it is not that clear for me. I also don't understand how you did the calculus to prove that the derivative of A(n) provides the moments of the sufficient statistics. $\endgroup$
    – sam
    Commented Mar 15, 2022 at 13:45
  • 1
    $\begingroup$ Parts that only contain $x$ that are separable from the others should be kept as is, as they form $h(x)$. The rest should go into the $\exp(\cdot)$ function, which can be done using the identity $c = \exp(\log c)$ for $c>0$. Per your second question, $\exp(-A(\eta))$ is what makes the integral of the rest normalize to 1, i.e., $\exp(A(\eta)) = \int h(x)\exp\{\eta^\top t(x) \}\,dx$, because every density should integrate to 1 by definition. Thus, we can use the identity $\frac{d}{d \eta} \log g(\eta) = \frac{g'(\eta)}{g(\eta)}$. $\endgroup$
    – Daeyoung
    Commented Mar 15, 2022 at 15:38
  • $\begingroup$ Thank you very much $\endgroup$
    – sam
    Commented Mar 15, 2022 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.