# Moments of the natural statistics of the normal gamma

I am trying to find the Moments of the natural statistics of the normal-gamma distribution. $$(X,T) - NormalGamma(\mu, \lambda,\alpha,\beta)$$ I found on its Wikipedia page that the moments of the natural statistics are: \begin{align} E(\ln T) &= \psi(\alpha) - \ln \beta,\\ E(T) &= \frac{\alpha}{\beta},\\ E(TX) &= \mu \frac{\alpha}{\beta},\\ E(T^2X) &= \frac{1}{\lambda} + \mu^2 \frac{\alpha}{\beta}. \end{align} The density function is $$f(x,t;\mu,\lambda,\alpha,\beta) = \frac{\beta^\alpha \sqrt{\lambda}}{\Gamma(\alpha)\sqrt{2\pi}} t^{\alpha-\frac{1}{2}} e^{-\beta t} \exp\left(-\frac{\lambda t(x-\mu)^2}{2} \right).$$ How are the above expectations derived?

• A crucial starting point is to define the pdf of the distribution you are interested in, given that there are usually multiple distributions sharing the same name, and to define $T$ and $X$ and the parameters. Absent that, your question does not have an answer. Commented Mar 14, 2022 at 21:33
• The pdf was already there, for T and X I thought they were clear for someone who knows this distribution
– sam
Commented Mar 15, 2022 at 14:15
• As the Wikipedia article states, these moments are readily derived from the moment generating function of $(X,T).$
– whuber
Commented Mar 15, 2022 at 14:23
• I know but I need the proof on how that was done
– sam
Commented Mar 15, 2022 at 14:28

First, note that normal-gamma distribution is an exponential family and has the following form: $$f(x;\eta) = h(x)\exp\{\color{blue}{\eta^\top t(x)} \color{red}{-A(\eta)} \},$$ where $$\eta$$ is the natural parameter, $$t(x)$$ is the sufficient statistic, $$h(x)$$ is the underlying measure, and $$A(\eta)$$ is the log-normalizing constant. What's interesting about the log-normalizing constant $$A(\eta)$$ is that its derivatives provide the moments of the sufficient statistic, i.e., \begin{align} \frac{d}{d\eta}A(\eta) &= \dfrac{d}{d\eta}\left(\log \int \exp\{\eta^\top t(x)\} h(x)\,dx \right)\\ &= \frac{\int t(x)\exp\{\eta^\top t(x)\}h(x)\,dx}{\int \exp\{\eta^\top t(x)\}h(x)\,dx}\\ &= \int t(x) \exp\{\eta^\top t(x)-A(\eta)\}h(x)\,dx\\ &= E(t(X)). \end{align}

Let's rewrite the density in terms of its natural parameter and sufficient statistic. Observe that \begin{align} f(X,T;\mu,\lambda,\alpha,\beta) &= \exp\Bigg\{\color{blue}{\left(\alpha-\frac{1}{2} \right)\log T -\left(\beta + \frac{\lambda \mu^2}{2} \right)T + \lambda \mu XT - \frac{\lambda}{2} TX^2} \\ &\quad \quad\color{red}{+\alpha \log \beta + \frac{1}{2}\log \lambda - \log \Gamma(\alpha) - \frac{1}{2}\log (2\pi)}\Bigg\}. \end{align} This part is merely matching the shape of the general form of an exponential family and a specific example. I colored the parts so each component is clearly separated from one another. Now, it's pretty clear from the blue part that $$\eta_1 = \alpha - \frac{1}{2},\;\eta_2 = -\beta - \frac{\lambda\mu^2}{2},\;\eta_3=\lambda\mu,\;\eta_4=-\frac{\lambda}{2},$$ and $$T_1 = \ln T,\;T_2=T,\; T_3=TX,\;T_4=TX^2,$$ so that $$\eta^\top t(x) = \eta_1T_1 + \eta_2T_2 + \eta_3T_3 + \eta_4T_4$$.

Next step is to reexpress $$A(\eta)$$ in terms of $$\eta$$. Note that $$\alpha= \eta_1+\frac{1}{2}$$, $$\beta=-\eta_2 + \frac{\eta_3^2}{4\eta_4}$$, $$\mu = -\frac{\eta_3}{2\eta_4}$$, and $$\lambda = -2\eta_4$$. Then, \begin{align} A(\eta)&= -\left(\eta_1+\dfrac{1}{2} \right)\log\left(\frac{\eta_3^2}{4\eta_4} - \eta_2 \right) -\frac{1}{2}\log(-2\eta_4) +\log\Gamma\left(\eta_1+\frac{1}{2}\right) + \frac{1}{2}\log(2\pi). \end{align}

From now on, it's good ol' differentiation: \begin{align} \frac{d}{d\eta_1}A(\eta) &= -\log\left(\frac{\eta_3^2}{4\eta_4} - \eta_2 \right)+ \psi\left( \eta_1+\frac{1}{2}\right) = \psi(\alpha)-\log(\beta),\\ \frac{d}{d\eta_2}A(\eta) &= \frac{\eta_1+\frac{1}{2}}{\frac{\eta_2^2}{4\eta_4}-\eta_2} = \frac{\alpha}{\beta},\\ \frac{d}{d\eta_3}A(\eta) &= -\left(\eta_1+\frac{1}{2} \right)\frac{\eta_3/(2\eta_4)}{\frac{\eta_3^2}{4\eta_4} - \eta_2} = \frac{\alpha \mu}{\beta},\\ \frac{d}{d\eta_4}A(\eta) &= -\left(\eta_1+\frac{1}{2} \right)\frac{-\frac{\eta_3^2}{4\eta_4^2}}{\frac{\eta_3^2}{4\eta_4} - \eta_2} - \frac{1}{2\eta_4} = \frac{\alpha\mu^2}{\beta} + \frac{1}{\lambda}. \end{align}

• Thank you for your help, but there are some points that I don't understand. The first one is how you managed to write the normal gamma in the exponential family format, you said it's merely matching but it is not that clear for me. I also don't understand how you did the calculus to prove that the derivative of A(n) provides the moments of the sufficient statistics.
– sam
Commented Mar 15, 2022 at 13:45
• Parts that only contain $x$ that are separable from the others should be kept as is, as they form $h(x)$. The rest should go into the $\exp(\cdot)$ function, which can be done using the identity $c = \exp(\log c)$ for $c>0$. Per your second question, $\exp(-A(\eta))$ is what makes the integral of the rest normalize to 1, i.e., $\exp(A(\eta)) = \int h(x)\exp\{\eta^\top t(x) \}\,dx$, because every density should integrate to 1 by definition. Thus, we can use the identity $\frac{d}{d \eta} \log g(\eta) = \frac{g'(\eta)}{g(\eta)}$. Commented Mar 15, 2022 at 15:38
• Thank you very much
– sam
Commented Mar 15, 2022 at 15:43