Consider $X \sim B(n,p)$ with pmf $P(X=x) = {{n}\choose{x}} p^x (1-p)^{n-x}$.

The general exponential form of an exponential family distribution is $p(x|\theta) = f(x) g(\theta) e^{\phi(\theta)^T T(x)}$. Writing the binomial distribution in this form, I get $f(x) = {n \choose x}, g(p)=(1-p)^n, \phi(\theta)=\log{\frac{p}{1-p}}, T(x) = x$.

In other words, $x$, the number of successes is a sufficient statistic for the probability of success on each Bernoulli trial, as is to be expected.

I am asked to find:

(1) the expected value of the sufficient statistic, $<T(x)>_{p(x|\theta)}$ (often called the "mean" or "moment" parameter of the distribution).

(2) the maximum likelihood value of the mean parameter in the above question

I have to be honest, I don't really understand the difference between the two questions. So far I have attempted this:

(1) $\theta=p$ so the likelihood $P(x|p)$ is the probability of getting p successes in n trials given a Bernoulli prob of p. i.e. $P(x|p) = {n \choose x} p^x(1-p)^x$.

Then $<T(x)> = \sum_x T(x)P(x|p) = \sum_x x {n \choose x} p^x(1-p)^x$.

At this point, I'm not sure what to do.

(2) I know that for MLE I should find the maximum of $log{L(p)}= log{P(x|p)} = \log{n \choose x} n+ n \log{1-p} + x \log{\frac{p}{1-p}}$. If I differentiate and set to zero, I get that $x = T(x) = np$ which seems right as this is the expected value for $X$ following a binomial distribution.

Am I at all on the right lines here?


Hint #0: computing the expectation of a sufficient statistic (of which there is an infinity if there is one) is not equivalent or even related to finding the maximum likelihood of $\theta$ (which for one thing depends on the choice of parameterisation $\theta$)

Hint #1: check Wikipedia for the expectation of a Binomial $\mathcal B(n,p)$ random variable

Hint #2: solve the likelihood equation in $p$, not in $x$

  • $\begingroup$ Thanks for the reply. To follow up on my original questions: (1) So I see from your Hint 0, I appear to have done this wrong as I suspected I had done. However, I am still not clear what I am supposed to be doing here? In my case, T(x)=x. How should I compute its expectation if not by the way above? (2) For the second question, I think my calculation is correct and I see that from the wording of the original question, I was supposed to be solving for the parameter and so I should have rearranged to p=n/x as you suggest. $\endgroup$ – user11128 Feb 15 '19 at 12:30
  • $\begingroup$ (1) compute the expectation of $X$ by identifying the distribution of $X$ (2) yes indeed $\endgroup$ – Xi'an Feb 15 '19 at 13:17
  • $\begingroup$ So basically $E(X) = \sum_x x P(X=x) = \sum_x x {n \choose x} p^x (1-p)^{n-x} =... =np$? $\endgroup$ – user11128 Feb 15 '19 at 13:38
  • $\begingroup$ This is presumably only the case because T(X) = X for this particular example? As you said, there are infinitely many possibilities for T(X) and so how could two statisticians be sure they get the same result? $\endgroup$ – user11128 Feb 15 '19 at 13:42
  • $\begingroup$ I do not understand your question: two different functions $T$ have different expectations in most cases, but this is of no consequence as a statistic is not an estimator by default. $\endgroup$ – Xi'an Feb 15 '19 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.