# Expectation of Sufficient Statistic for Beta Distribution

I am looking at question 1b of the following notes: http://www.gatsby.ucl.ac.uk/teaching/courses/ml1/asst1.pdf

In 1a, I have shown that the Beta distribution has a density that can be written in the form $$g(\boldsymbol{\theta}) f(\textbf{x}) e^{\phi(\boldsymbol{\theta})^T T(\textbf{x})}$$ where:

$$g(\boldsymbol{\theta}) = \frac{1}{B(\alpha,\beta)}$$

$$f(x) = \frac{1}{x(1-x)}$$

$$\phi(\boldsymbol{\theta}) = \begin{pmatrix} \alpha \\ \beta \end{pmatrix}$$

$$T(x) = \begin{pmatrix} \log{x} \\ \log{(1-x)} \end{pmatrix}$$

Now, according to this thread, I can calculate the expectation of the sufficient statistic, $$E[T(x)]$$, by rewriting the density as

$$f(\textbf{x}) e^{\phi(\boldsymbol{\theta})^T T(\textbf{x})} e^{-A(\boldsymbol{\theta})}$$

and then using $$E[T(x)]_i = \frac{A'(\boldsymbol{\theta})_i}{\phi'(\boldsymbol{\theta})_i}$$

To do this, we define $$A(\theta) = - \log{g(\theta)} = \log{B(\alpha,\beta)}$$ but then I become uncertain of what to do. My attempt so far looks like:

$$A'(\theta) = \frac{1}{B(\alpha,\beta)} \begin{pmatrix} B_\alpha \\ B_\beta \end{pmatrix}$$ where $$B_\alpha = \frac{\partial B}{\partial \alpha}$$ etc. I also find $$\phi'(\theta) = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$$.

I'm pretty sure I've made a mistake with the calculation of $$phi'(\theta)$$ since the vector of ones seems wrong, and yet $$\phi'(\theta) = \frac{\partial}{\partial \theta_i} \phi(\theta) = \begin{pmatrix} \frac{\partial}{\partial \alpha} \\ \frac{\partial}{\partial \beta} \end{pmatrix} \phi(\alpha, \beta)$$. Is this going to become a 2x2 matrix?

I have managed to get this technique to work for distributions such as binomial and Poisson where the sufficient statistic is a scalar. I am getting confused in this case where we have a vector of sufficient statistics.

• Please make the question self-contained rather than expecting the reader to follow a web link. – Xi'an May 8 '19 at 8:14

The thread you provide only operates for one-dimensional exponential families. In general, when the exponential family is written wrt its natural parameter: $$f(x) e^{\theta^T T(x)} e^{-A(\boldsymbol{\theta})}$$ the mean of the associated (sufficient) statistic is $$\Bbb E_\boldsymbol\theta[T(X)] = \nabla A(\boldsymbol{\theta})$$ Since this is the case for the Beta distribution, namely $$\phi( \boldsymbol\theta)=\boldsymbol\theta$$, $$\nabla A(\boldsymbol{\theta})=\frac{1}{B(\alpha,\beta)} \begin{pmatrix} \frac{\partial B(\alpha,\beta)}{\partial\alpha} \\ \frac{\partial B(\alpha,\beta)}{\partial\beta} \end{pmatrix}$$
• It seems that we cannot always have $\phi(\theta) =\theta$ however. For example, with the gamma distribution, I find $\phi(\theta) =( \alpha - 1, - \beta) ^T$. Under these circumstances, if I follow through the algebra, we would have $(\nabla \phi) E[T(x)] = \nabla A$. Is that correct? – user11128 May 26 '19 at 9:14
• You can always reparameterise $\phi(\theta)$ as the default $\theta$. – Xi'an May 26 '19 at 10:22