Moments of the natural statistics of the normal gamma

Question

I am trying to find the Moments of the natural statistics of the normal-gamma distribution. $$(X,T) - NormalGamma(\mu, \lambda,\alpha,\beta)$$ I found on its Wikipedia page that the moments of the natural statistics are: $$ \begin{align} E(\ln T) &= \psi(\alpha) - \ln \beta,\\ E(T) &= \frac{\alpha}{\beta},\\ E(TX) &= \mu \frac{\alpha}{\beta},\\ E(T^2X) &= \frac{1}{\lambda} + \mu^2 \frac{\alpha}{\beta}. \end{align} $$ The density function is $$ f(x,t;\mu,\lambda,\alpha,\beta) = \frac{\beta^\alpha \sqrt{\lambda}}{\Gamma(\alpha)\sqrt{2\pi}} t^{\alpha-\frac{1}{2}} e^{-\beta t} \exp\left(-\frac{\lambda t(x-\mu)^2}{2} \right). $$ How are the above expectations derived?

Answer 1

First, note that normal-gamma distribution is an exponential family and has the following form: $$ f(x;\eta) = h(x)\exp\{\color{blue}{\eta^\top t(x)} \color{red}{-A(\eta)} \}, $$ where $\eta$ is the natural parameter, $t(x)$ is the sufficient statistic, $h(x)$ is the underlying measure, and $A(\eta)$ is the log-normalizing constant. What's interesting about the log-normalizing constant $A(\eta)$ is that its derivatives provide the moments of the sufficient statistic, i.e., $$ \begin{align} \frac{d}{d\eta}A(\eta) &= \dfrac{d}{d\eta}\left(\log \int \exp\{\eta^\top t(x)\} h(x)\,dx \right)\\ &= \frac{\int t(x)\exp\{\eta^\top t(x)\}h(x)\,dx}{\int \exp\{\eta^\top t(x)\}h(x)\,dx}\\ &= \int t(x) \exp\{\eta^\top t(x)-A(\eta)\}h(x)\,dx\\ &= E(t(X)). \end{align} $$

Let's rewrite the density in terms of its natural parameter and sufficient statistic. Observe that $$ \begin{align} f(X,T;\mu,\lambda,\alpha,\beta) &= \exp\Bigg\{\color{blue}{\left(\alpha-\frac{1}{2} \right)\log T -\left(\beta + \frac{\lambda \mu^2}{2} \right)T + \lambda \mu XT - \frac{\lambda}{2} TX^2} \\ &\quad \quad\color{red}{+\alpha \log \beta + \frac{1}{2}\log \lambda - \log \Gamma(\alpha) - \frac{1}{2}\log (2\pi)}\Bigg\}. \end{align} $$ This part is merely matching the shape of the general form of an exponential family and a specific example. I colored the parts so each component is clearly separated from one another. Now, it's pretty clear from the blue part that $$ \eta_1 = \alpha - \frac{1}{2},\;\eta_2 = -\beta - \frac{\lambda\mu^2}{2},\;\eta_3=\lambda\mu,\;\eta_4=-\frac{\lambda}{2}, $$ and $$ T_1 = \ln T,\;T_2=T,\; T_3=TX,\;T_4=TX^2, $$ so that $\eta^\top t(x) = \eta_1T_1 + \eta_2T_2 + \eta_3T_3 + \eta_4T_4$.

Next step is to reexpress $A(\eta)$ in terms of $\eta$. Note that $\alpha= \eta_1+\frac{1}{2}$, $\beta=-\eta_2 + \frac{\eta_3^2}{4\eta_4}$, $\mu = -\frac{\eta_3}{2\eta_4}$, and $\lambda = -2\eta_4$. Then, $$ \begin{align} A(\eta)&= -\left(\eta_1+\dfrac{1}{2} \right)\log\left(\frac{\eta_3^2}{4\eta_4} - \eta_2 \right) -\frac{1}{2}\log(-2\eta_4) +\log\Gamma\left(\eta_1+\frac{1}{2}\right) + \frac{1}{2}\log(2\pi). \end{align} $$

From now on, it's good ol' differentiation: $$ \begin{align} \frac{d}{d\eta_1}A(\eta) &= -\log\left(\frac{\eta_3^2}{4\eta_4} - \eta_2 \right)+ \psi\left( \eta_1+\frac{1}{2}\right) = \psi(\alpha)-\log(\beta),\\ \frac{d}{d\eta_2}A(\eta) &= \frac{\eta_1+\frac{1}{2}}{\frac{\eta_2^2}{4\eta_4}-\eta_2} = \frac{\alpha}{\beta},\\ \frac{d}{d\eta_3}A(\eta) &= -\left(\eta_1+\frac{1}{2} \right)\frac{\eta_3/(2\eta_4)}{\frac{\eta_3^2}{4\eta_4} - \eta_2} = \frac{\alpha \mu}{\beta},\\ \frac{d}{d\eta_4}A(\eta) &= -\left(\eta_1+\frac{1}{2} \right)\frac{-\frac{\eta_3^2}{4\eta_4^2}}{\frac{\eta_3^2}{4\eta_4} - \eta_2} - \frac{1}{2\eta_4} = \frac{\alpha\mu^2}{\beta} + \frac{1}{\lambda}. \end{align} $$