0
$\begingroup$

Let the random variable $X$ follow the distribution:

$$ f(x;\theta) = \theta^2(x+1)(1-\theta)^x$$

where $x$ takes values in $[0, \infty)$ and $\theta$ in $[0, 1]$. The likelihood is defined as:

$$\mathcal{L}(x \mid \theta) = p_\theta(x)$$

and for $N$ i.i.d observations:

$$\mathcal{L}(\theta) = \prod_{i=1}^N p_\theta(x_i)$$

Due to the monoticity of the $\ln$ function, we can instead maximize the log-likelihood:

$$\ln(\mathcal{L}(\theta)) = \ln \left( \prod_{i=1}^N p_\theta(x_i) \right) = \sum_{i=1}^{N} \ln p_\theta(x_i) = \sum_{i=1}^{N} x_i\ln \left[\theta^2 (x+1)(1-\theta)\right]$$

The derivative of the likelihood with respect to $\theta$ is:

$$\frac{d\ln(\mathcal{L}(\theta))}{d\theta} = \sum_{i=1}^{N} x_i \frac{d}{d\theta} \ln\left[\theta^2 (x+1)(1-\theta)\right] = \sum_{i=1}^{N} x_i \frac{d}{d\theta} \left[ \ln(\theta^2) + \ln(x_i + 1) + \ln(1-\theta)\right]$$

Taking the derivatives of the terms inside the brackets we obtain:

$$\sum_{i=1}^{N} x_i \left(\frac{2}{\theta} - \frac{1}{1-\theta}\right)$$

and setting this to zero leads to:

$$\sum_{i=1}^{N} x_i \frac{2}{\theta} = \sum_{i=1}^{N} x_i \frac{1}{1-\theta} \Longleftrightarrow \frac{2}{\theta}\sum_{i=1}^n x_i = \frac{1}{1-\theta} \sum_{i=1}^n x_i \Longleftrightarrow \frac{2}{\theta} = \frac{1}{1-\theta}$$

Solving the last equations leads to $\hat{\theta}=\frac{2}{3}$.

Is it possible for the MLE to be independent of the observations ($x_i$ are not involved in calculation of $\hat{\theta}$)? If yes, is there something in the form of $f(x;\theta)$ that indicates that?

$\endgroup$
2
  • 1
    $\begingroup$ Your calculation of the likelihood equation is incorrect. It is easiest to work with a single observation when calculating the MLE. $\endgroup$
    – AdamO
    Commented Oct 17, 2022 at 18:24
  • $\begingroup$ I think the calculation with logarithm has a mistake: $\ln(p_\theta(x)) = 2\ln \theta + \ln(x+1) + x \ln(1-\theta)$ $\endgroup$
    – Kota Mori
    Commented Oct 17, 2022 at 18:29

1 Answer 1

1
$\begingroup$

First, it should be noted that your computation is incorrect: you cannot distribute the $x_i$ across all of the terms. The log-likelihood function should instead look like the following: $$\sum_{i=1}^n 2 \log \theta + \log(x_i+1) + x_i\log(1-\theta)$$ so that the derivative of the log-likelihood function is $$\sum_{i=1}^n \frac2{\theta} - \frac{x_i}{1-\theta} = 0$$ or solving, $$\hat\theta = f(\bar x)$$ where $f$ is the function defined such that $f^{-1}(\theta) = \frac{2(1-\theta)}{\theta}$ and $\bar x$ is the sample mean. This does indeed depend on the data.

More broadly speaking, let's take a step back and ask conceptually what it would mean for the MLE to not depend on the data. Intuitively, this amounts to claiming that no matter what the data says, our best guess of the true parameter is the same. Does this make sense? If this was true, what would even be the point of looking at the data?

$\endgroup$
2
  • $\begingroup$ Oops, thank you for pointing out the fallacy in the calculation. Your last paragraph was my motivation for asking this question. I found it meaningless that MLE is independent of the data (of course I had made an incorrect computation). Also, is there any way that can can prove that indeed the MLE we calculate by setting the derivative to zero is a maximum? I mean all the sources I have seen they don't further examine whether this critical point is a minimum or maximum (e.g. taking the second derivative). $\endgroup$
    – ado sar
    Commented Oct 17, 2022 at 18:41
  • 1
    $\begingroup$ @adosar the uniqueness and existence of an MLE depends on a lot. In elementary probability courses, you usually deal with regular exponential families and parameters that do not lie on the boundary of the parameter space. The proofs that these methods are well behaved are extremely complicated. But one does not need to go far in statistics to find models that deal with this problem all the time. $\endgroup$
    – AdamO
    Commented Oct 17, 2022 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.