3
$\begingroup$

I have a particular PDF with two parameters, specified as:

$$\alpha \beta e^{-\beta x}(1 - e^{-\beta x})^{\alpha - 1}, \alpha > 0, \beta > 0, x_i > 0$$

Given a random iid sample $(x_1, \dots, x_n)$ from the above, I can then write the likelihood function as:

$$L(\alpha, \beta \mid x_1, \dots, x_n) = (\alpha \beta)^n \prod_{i=1}^n e^{-\beta x}(1-e^{-\beta x})^{\alpha-1}$$

Taking the log thereof to make things slightly easier, we then have:

$$\log(L) = n \log(\alpha) + n \log(\beta) - \beta \sum_{i=1}^n x_i + (\alpha-1) \sum_{i=1}^n \log(1 - e^{-\beta x_i})$$

Taking the derivative with respect to $\alpha$ and setting to zero:

$$\frac{\partial}{\partial \alpha} \log(L) = 0 = \frac{n}{\alpha} + \sum_{i=1}^n \log(1-e^{-\beta x_i})$$ $$\hat{\alpha} = \frac{-n}{\sum_{i=1}^n \log(1 - e^{-\beta x_i})}$$

Then, plugging this back into the likelihood and repeating the process for $\beta$, I get something extremely messy and would have to numerically maximize. Did I screw up some math somewhere? No closed form implies that I will need numerical maximization for $\beta$...

$\endgroup$
3
$\begingroup$

The (correct) log-likelihood is

$$\log(L) = n \log(\alpha) + n \log(\beta) - \beta \sum_{i=1}^n x_i + (\alpha-1) \sum_{i=1}^n \log(1 - e^{-\beta x_i})$$

The f.o.c for $\alpha$ is

$$\hat{\alpha} = \frac{n}{-\sum_{i=1}^n \log(1 - e^{-\beta x_i})}$$

Plugging this into the log-likelihood we get

$$\log(L \mid \hat \alpha) = n \log\left(\frac{n}{-\sum_{i=1}^n \log(1 - e^{-\beta x_i})}\right) + n \log(\beta) - \beta \sum_{i=1}^n x_i + \left(\frac{n}{-\sum_{i=1}^n \log(1 - e^{-\beta x_i})}-1\right) \sum_{i=1}^n \log(1 - e^{-\beta x_i})$$

$$ = const. -n\log \left(-\sum_{i=1}^n \log(1 - e^{-\beta x_i})\right) + n \log(\beta) - \beta \sum_{i=1}^n x_i - \sum_{i=1}^n \log(1 - e^{-\beta x_i})$$

I wouldn't call the above,or the consequent derivative, "extremely messy", and yes you will need numerical maximization, which is the rule rather than the exception with ML estimation.

$\endgroup$
  • $\begingroup$ I suppose 'extremely messy' is just my way of saying that you cannot easily maximize using the derivative (as you might normally expect from a textbook example). Just because I'm curious about your last remark: is this (numerical maximization being the rule) because of the frequency with which one encounters less convenient likelihood functions? $\endgroup$ – PatternMatching Mar 6 '15 at 20:48
  • $\begingroup$ @PatternMatching Yes, that's an observation based on experience. For example, almost any likelihood that includes a cumulative distribution function (say for a censored sample) will need numerical maximization. $\endgroup$ – Alecos Papadopoulos Mar 6 '15 at 21:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.