4
$\begingroup$

I am using this formulation of the exponential family : $$ \large f_{X}(x;\boldsymbol{\eta})=h(x) \exp \left(\boldsymbol{\eta} \cdot \mathbf{T}(x)-A(\boldsymbol{\eta})\right) $$ The gamma distribution is given as : $$ \large f _{X}(x\,;\alpha, \beta) = \frac{{\beta^{\alpha} x ^{\alpha - 1}\exp(-\beta x)}}{\Gamma(\alpha)} $$ Doing some algebraic manipulations on this PDF, we can show that : $$ \boldsymbol{\eta} = \begin{bmatrix} \eta _{1} \\ \eta _{2} \end{bmatrix} = \begin{bmatrix} \alpha - 1 \\ -\beta \end{bmatrix} $$ Similarly for the log partition function : $$ A(\boldsymbol{\eta}) = \ln(\Gamma(\eta _{1} + 1)) - (\eta _{1} + 1)\ln(-\eta _{2}) $$ Now the mean of the distribution should be calculated as follows : $$ \boldsymbol{\mu} = \nabla _{\boldsymbol{\eta}} A(\boldsymbol{\eta}) $$ Further calculating, the "mean vector" can be found to be : $$ \boldsymbol{\mu} = \begin{bmatrix}\psi(\alpha) - \ln(\beta) \\ \alpha / \beta \end{bmatrix} $$ Where $\psi$ is the digamma function.

However, this implies that the mean of the gamma distribution will be a two dimensional vector which is blatantly not the case!
Can anyone point out what exactly am I doing wrong here?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

$\boldsymbol{\mu}$ is not the expected value of an exponential family distribution. It is the expected value of the distribution's sufficient statistic $\mathbf{T}(X)$.
For a $\mathop{\mathrm{Gamma}}\left(\alpha, \beta\right)$-distributed random variable $X$ we have $\mathbf{T}(X) = \left(\ln\left(X\right), X\right)^\top$ and hence $\mathop{\mathbb E}\left(X\right) = \boldsymbol{\mu}_2 = \alpha/\beta$.

Improve this answer
$\endgroup$
6
  • $\begingroup$ So is $\boldsymbol \mu$ not the mean then? Moreover, is the mean defined to be the expectation of the sufficient statistic or the expectation of the random variable? $\endgroup$
    – Sagnik Taraphdar
    Commented Jan 15 at 5:12
  • $\begingroup$ @SagnikTaraphdar $\boldsymbol \mu$ is the mean of $\mathbf T(X)$. The terms "mean" and "expected value" of a random variable are synonymous. The mean of a distribution $\mathbb P$ is defined as the expected value of a $\mathbb P$-distributed random variable. For a real-valued random variable $X$ with probability density function $f_X$ (w.r.t. the Lebesgue measure) it is given by $\int_{\mathbb R} x f_X(x) \, \mathrm dx$. $\endgroup$
    – statmerkur
    Commented Jan 15 at 13:25
  • $\begingroup$ I understand that but my only remaining query was this : is the mean of the sufficient statistic, here denoted as $\mathbf \mu$ , not the same as the mean of the distribution? $\endgroup$
    – Sagnik Taraphdar
    Commented Jan 15 at 13:27
  • $\begingroup$ @SagnikTaraphdar $\boldsymbol \mu = \mathbb E[\mathbf T(X)] = \mathbb E[\left(\ln\left(X\right), X\right)^\top] = (\mathbb E[\ln(X)], \mathbb E[X])^\top$. But the mean of a $\mathop{\mathrm{Gamma}}\left(\alpha, \beta\right)$ distribution is the expected value of a $\mathop{\mathrm{Gamma}}\left(\alpha, \beta\right)$-distributed random variable $X$, i.e. $\mathbb E[X]$. Clearly, $(\mathbb E[\ln(X)], \mathbb E[X])^\top$ and $\mathbb E[X]$ are not the same. $\endgroup$
    – statmerkur
    Commented Jan 15 at 13:44
  • $\begingroup$ Thanks for the clarification! I was just confused as to how the mean of the distribution was related to the log-partition function. I didn't realize it was actually the mean of the sufficient statistic that's related to the log-partition function. $\endgroup$
    – Sagnik Taraphdar
    Commented Jan 15 at 13:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.