Given a random sample $X_1, X_2,..., X_n$ from Bernoulli distribution. The log-likelihood function is:

$\mathcal{L}(\theta) = \sum_1^n x_i^*\log{\theta} + (n - \sum_1^n x_i^*)\log{(1-\theta)}$

Score function:

$\frac{\partial \mathcal{L}}{\partial \theta} = \frac{\sum_1^n x_i^*}{\theta} - \frac{n - \sum_1^n x_i^*}{1 - \theta} \; \forall \theta \in (0,1)$

By solving the score equation $\frac{\partial \mathcal{L}}{\partial x} = 0 $ we get $\theta = \bar{x}$ is a potential maximum likelihood estimate for $\theta$.

We need to verify that the second derivative is negative at $\theta = \bar{x}$.

$\frac{\partial^2 \mathcal{L}}{\partial^2 \theta} = -\frac{\sum_1^n x_i^*}{\theta^2} - \frac{n - \sum_1^n x_i^*}{(1 - \theta)^2} \; \forall \theta \in (0,1)$

$\frac{\partial^2 \mathcal{L}}{\partial^2 \theta}\rvert_{\theta = \bar{x}} = -\frac{\sum_1^n x_i^*}{\bar{x}^2} - \frac{n - \sum_1^n x_i^*}{(1 - \bar{x})^2}$

At this step, some people will conclude immediately that the second derivative evaluated at maximum likelihood estimate is negative, so $\bar{X}$ is maximum likelihood estimator for $\theta$.

However, do I have to say that the second derivative is negative under the condition that $\bar{x} \neq 0$ and $ 1 - \bar{x} \neq 0$? Or is this constraint implied from the beginning? I just want everything I write in my notes/slides to be completely precise.

I know that if $\bar{x} = n$ or $\bar{x} = 0$, the likelihood function will behave differently as it will have a reflection point, but the maximum likelihood estimate for these situations is still $\bar{x}$. Of course, we cannot use the method shown above to derive the maximum likelihood estimate.

  • 2
    $\begingroup$ $\theta = 0$ and $\theta = 1$ are in fact permissible values for the probability parameter of a Bernoulli distribution; knowing this should help resolve your difficulties. $\endgroup$
    – jbowman
    Commented Jun 20, 2022 at 11:37
  • $\begingroup$ @jbowman Thanks for your comment. I know that $\theta \in [0,1]$, but I excluded 0 and 1 because the first derivative of log-likelihood function is not well-defined at these values of $\theta$, but there is still a possibility that you will get a sample with variance equal to 0 even if $\theta \neq 0$ and $\theta \neq 1$. I want to know if I can conclude that the second derivative is negative if and only if $\bar{x}$ and $1 - \bar{x}$ are different from 0. $\endgroup$
    – user340483
    Commented Jun 20, 2022 at 12:08
  • 1
    $\begingroup$ math.stackexchange.com/questions/410364/… essentially is your question. $\endgroup$
    – jbowman
    Commented Jun 20, 2022 at 12:36
  • $\begingroup$ All will become clear (I believe) when you plot an example of such a likelihood function: namely, a non-negative linear combination of $\log \theta$ and $\log(1-\theta)$ defined on the interval $(0,1)$ (and prolonged to a definition on $[0,1]$ using the extended reals, if you like). $\endgroup$
    – whuber
    Commented Jun 20, 2022 at 12:51


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.