Estimation of parameters (Likelihood/Moments) and Bias

Question

$X_1,\dots,X_n$ are a random sample of X. I want to estimate $\theta$ and check bias for the following density function:

$f(x|\theta) = \frac{2x}{\theta^2}, 0 \leq x \leq \theta$ and $\theta > 0$

At first, I did it with the Moments method:

$\hat{\alpha} = \frac{1}{n} \sum_{i=1}^{n} X_i$

$E[X] = \int_{0}^{\theta} \frac{2{x}^2}{\theta^2}dx = \theta$

Then: $\hat{\theta} = \frac{1}{n} \sum_{i=1}^{n} X_i$

And $E[\hat{\theta}] = E[\frac{1}{n} \sum_{i=1}^{n} X_i] = \frac{1}{n}\sum_{i=1}^{n}E[X_i] = \frac{n}{n}\theta$

So we can conclude that the estimator found by the moments method was unbiased.

Now, I tried to do the same using maximum likelihood, but I found the following:

$L(\theta|x) = \frac{2}{\theta^{2n}}\prod_{i=1}^{n}X_i$

Taking log on both sides:

$l(\theta|x) = 2log(\sum_{i=1}^{n} x_i) - 2nlog(\theta)$

The estimate using maximum likelihood,

$\frac{dl(\theta|x)}{d\theta} = 0$

But then I find:

$\frac{n}{\theta} = 0$

(the first term is 0 when derived, as it is not a function of $\theta$)

Which makes no sense. What did I do wrong here? Or is there an algebraic trick I am missing to calculate this estimator through maximum likelihood?

Answer 1

The likelihood is a function. A function is not automatically maximized at a turning point. (E.g. consider -- where's the mode of an exponential density?)

Don't just start taking derivatives without thinking about what the shape of the likelihood is and where the parameters live.

Instead, first sketch the likelihood (or its log if you prefer, or even both). Be very careful about the limits. (Think about inequalities relating data and parameters.)

If you have two observations, $x_1=1.5$ and $x_2=2.4$, what's the likelihood at at $\theta=3$? At $\theta=2.5$? At $\theta=2$? ... then if need be try this larger sample: $(2.13, 1.09, 2.43, 2.33, 1.5, 2.39, 1.61, 1.89, 0.62, 1.26)$ with those same questions.

---

Responses to comments

So, it seems my log likelihood function gets lower as my $\theta$ goes higher. As I want to maximize log likelihood, I get the minimum possible value of $\theta$. Due to my restrictions, it will be $max(x_1,\dots,x_i)$. This will be my maximum likelihood estimate.

This is the right general idea.

(Working with indicator functions is the easy way to do this sort of thing formally, but on simple problems like this one we can manage perfectly well without it.)

if the above is correct, does this mean that the estimator I found with the moments method doesn't make sense, as there will be $x_i$ bigger than $\hat{\theta}$?

Method of moments can yield parameter estimates that are impossible (such as giving estimates where not all observations lie inside $(0,\hat{\theta})\,$).

It's not guaranteed that you will get $\hat{\theta}<x_{(n)}$ (i.e. less than the largest observation), but it is a possibility.