# Estimation of parameters (Likelihood/Moments) and Bias

$X_1,\dots,X_n$ are a random sample of X. I want to estimate $\theta$ and check bias for the following density function:

$f(x|\theta) = \frac{2x}{\theta^2}, 0 \leq x \leq \theta$ and $\theta > 0$

At first, I did it with the Moments method:

$\hat{\alpha} = \frac{1}{n} \sum_{i=1}^{n} X_i$

$E[X] = \int_{0}^{\theta} \frac{2{x}^2}{\theta^2}dx = \theta$

Then: $\hat{\theta} = \frac{1}{n} \sum_{i=1}^{n} X_i$

And $E[\hat{\theta}] = E[\frac{1}{n} \sum_{i=1}^{n} X_i] = \frac{1}{n}\sum_{i=1}^{n}E[X_i] = \frac{n}{n}\theta$

So we can conclude that the estimator found by the moments method was unbiased.

Now, I tried to do the same using maximum likelihood, but I found the following:

$L(\theta|x) = \frac{2}{\theta^{2n}}\prod_{i=1}^{n}X_i$

Taking log on both sides:

$l(\theta|x) = 2log(\sum_{i=1}^{n} x_i) - 2nlog(\theta)$

The estimate using maximum likelihood,

$\frac{dl(\theta|x)}{d\theta} = 0$

But then I find:

$\frac{n}{\theta} = 0$

(the first term is 0 when derived, as it is not a function of $\theta$)

Which makes no sense. What did I do wrong here? Or is there an algebraic trick I am missing to calculate this estimator through maximum likelihood?

• The likelihood is a function. A function is not automatically maximized at a turning point. (E.g. consider -- where's the mode of an exponential density?) Don't just blithely start taking derivatives without thinking about what the shape of the likelihood is and where the parameters live. Instead, first sketch the likelihood (or its log if you prefer, or even both). Be very (very) careful about the limits. (Think about inequalities relating data and parameters.) Oct 20 '16 at 1:10
• If you have two observations, $x_1=1.5$ and $x_2=2.4$, what's the likelihood at at $\theta=3$? At $\theta=2.5$? At $\theta=2$? ... then if need be try this larger sample: $(2.13, 1.09, 2.43, 2.33, 1.5, 2.39, 1.61, 1.89, 0.62, 1.26)$ with those same questions. Oct 20 '16 at 1:19
• So, it seems my log likelihood function gets lower as my $\theta$ goes higher. As I want to maximize log likelihood, I get the minimum possible value of $\theta$. Due to my restrictions, it will be $max(x_1,\dots,x_i)$. This will be my maximum likelihood estimate. Is this right? Oct 20 '16 at 19:25
• Just adding, if the above is correct, does this mean that the estimator I found with the moments method doesn't make sense, as there will be $x_i$ bigger than $\hat{\theta}$? Oct 20 '16 at 19:39

The likelihood is a function. A function is not automatically maximized at a turning point. (E.g. consider -- where's the mode of an exponential density?)

Don't just start taking derivatives without thinking about what the shape of the likelihood is and where the parameters live.

Instead, first sketch the likelihood (or its log if you prefer, or even both). Be very careful about the limits. (Think about inequalities relating data and parameters.)

If you have two observations, $x_1=1.5$ and $x_2=2.4$, what's the likelihood at at $\theta=3$? At $\theta=2.5$? At $\theta=2$? ... then if need be try this larger sample: $(2.13, 1.09, 2.43, 2.33, 1.5, 2.39, 1.61, 1.89, 0.62, 1.26)$ with those same questions.

So, it seems my log likelihood function gets lower as my $\theta$ goes higher. As I want to maximize log likelihood, I get the minimum possible value of $\theta$. Due to my restrictions, it will be $max(x_1,\dots,x_i)$. This will be my maximum likelihood estimate.
if the above is correct, does this mean that the estimator I found with the moments method doesn't make sense, as there will be $x_i$ bigger than $\hat{\theta}$?
Method of moments can yield parameter estimates that are impossible (such as giving estimates where not all observations lie inside $(0,\hat{\theta})\,$).
It's not guaranteed that you will get $\hat{\theta}<x_{(n)}$ (i.e. less than the largest observation), but it is a possibility.