MLE for Independent Exponentials

Here is the problem statement:

Let $(X_i,Y_i)$ be a random sample from a distribution with pdf $$f(x,y;\theta)= > \frac{1}{\theta^3}\exp\left(\frac{-x}{\theta}-\frac{y}{\theta^2}\right)\qquad > 0<x, 0<y.$$ (a) Find the MLE for $\theta$.

(b) Give the asymptotic distribution of $\sqrt{n}(\hat{\theta}-\theta).$

Here are my thoughts: We basically have two random samples, $X\sim \exp(\theta)$ and $Y\sim \exp(\theta^2)$. This pdf is the joint pdf for $X$ and $Y$. The parameter has no impact on the support of the variables $X$ and $Y$, so calculus should suffice for this problem. $$L(x,y;\theta) = \prod_{i=1}^n f(x,y;\theta) = \frac{1}{\theta^{3n}}\exp\left(\frac{-1}{\theta^2}\sum_{i=1}^n \theta x_i+y_i\right),$$ so $$l(x,y;\theta) = -3n\ln(\theta)-\frac{1}{\theta^2}\sum_{i=1}^n \theta x_i + y_i,$$ which has partial $\theta$ derivative $$\frac{-3n\theta^2+\sum_{i=1}^n \theta x_i +2y_i}{\theta^3}.$$ Equating to zero and solving for $\theta$ requires solving a quadratic in $\theta$.

The second $\theta$ derivative is $$\frac{3n\theta^2 - \left(\sum_{i=1}^n 2\theta x_i + 6y_i\right)}{\theta^4}.$$ The solutions to the quadratic are $$\theta = \frac{\sum_{i=1}^n x_i\pm \sqrt{\left(\sum_{i=1}^nx_i\right)^2+24n\sum_{i=1}^n y_i}}{6n}.$$ Is there no easier way to do the problem? Am I forced to make this substitution to find the maximum?

• Some brackets inside your SUMS would make your notation clearer. Your derivation so far is correct. You need to break the sum of $(x_i+y_i)$ into separate terms for x and y. Then you can solve the quadratic in $\theta$. – wolfies Mar 2 '16 at 4:10
• The roots are not convenient to write and the second derivative is not obviously nonpositive, so I feel I am still stuck on deciding if my roots are minima. Substitution seems nightmarish. – Hamilton Mar 2 '16 at 4:56
• It's just occurred to me: Is the MLE of a product the product of the MLE's? That would save quite a headache with algebra. – Hamilton Mar 2 '16 at 5:10
• Consider $\bar{x}=1$ and $\bar{y}=2$ and draw the log-likelihood(with $n=4$ if you need it, though $n$ is actually immaterial, you can pull a factor of $n$ out of every term). [You may find it easier to identify what's going on if you plot log-likelihood vs 1/theta but YMMV.] – Glen_b Mar 2 '16 at 5:21
• If you think about what's going on with the two solutions, you should be able to write a single solution down (the other you can eliminate for two reasons, one trivial, one still fairly simple, either of which is sufficient to rule it out). You should be able to then argue why the remaining turning point must indeed be a maximum. You can also simplify the formula a little though it's not necessary. – Glen_b Mar 2 '16 at 5:29

I hesitated to 'answer' the question before, because it was not clear if this was a homework assignment, ... but as you have now answered it yourself (and neatly so), I thought I might briefly reply to the question posed:

$$\text{"Is there no easier way to do the problem?"}$$

In exactly this vein, my co-author, Murray Smith, and I sought to automate these sorts of MLE problems, symbolically and exactly, in precisely the way you have proceeded - step by step - but getting the computer to do the grunt work. See, for instance:

• Rose, C and Smith, M.D. (2000), Symbolic maximum likelihood estimation with Mathematica, Journal of the Royal Statistical Society, Series D: The Statistician, 49(2), 2000, 229-240. Download available: here

A more sophisticated version of same was then built into the mathStatica software package which we later developed.

GIVEN: $$(X,Y)$$ have a bivarate Exponential distribution, with parameter $$\theta >0$$, with joint pdf $$f(x,y)$$:

(source: tri.org.au)

Activate the special SuperLog function:

(source: tri.org.au)

For a random sample of size $$n$$ drawn on $$(X,Y)$$, the log-likelihood function for parameter $$\theta$$ is:

(source: tri.org.au)

The score function is the gradient of the log-likelihood with respect to $$\theta$$:

(source: tri.org.au)

Setting the score to zero and solving for $$\theta$$ corresponds to the first-order condition:

(source: tri.org.au)

We require the second (positive) solution. As for second-order conditions, the Hessian, evaluated at the optimal solution, is:

(source: tri.org.au)

... which is strictly negative, given that $$X>0$$ and $$Y>0$$.

• I was thinking more about it today and I realized that the first derivative is sufficient. Once I have the solutions (which correspond to roots of a concave parabola), I can analyze the behavior near a zero to classify one solution as a maximum and one as a minimum. The second derivative IS unnecessary. Thanks for this code. – Hamilton Mar 2 '16 at 22:49