# How to show the rate of convergence of this maximum likelihood estimator is $n^{-1}$?

Suppose I have data $$\{X_i\}_{i=1}^{n}\sim \text{Uniform}[0,\theta_0]$$. We know that the maximum likelihood estimator of $$\theta_0$$ is $$\widehat{\theta}=\max\{X_1,...,X_n\}$$. How to show that $$\widehat{\theta}$$ converges to $$\theta_0$$ at rate $$n^{-1}$$? Thanks!

## 1 Answer

When $$(X_i)$$ is a sequence of uniformly distributed variables $$\mathcal{U}_{[0,\theta_0]}$$ we can derive the distribution of the MLE $$\hat \theta = \max X_i$$.
We will show that $$n( \theta_0 - \hat \theta)$$ converge towards a non trivial (i.e. not $$0$$) distribution, which means that the rate of convergence of the MLE is $$n^{-1}$$.

First note that since $$\max X_i \leq \theta_0$$, $$n( \theta_0 - \hat \theta) \geq 0$$.

Then, for $$t \geq 0$$,

\begin{align*} \mathbb P\left( n(\theta_0 -\hat \theta) \leq t \right) &=1- \mathbb P\left( \hat \theta \leq \theta_0 - \frac{t}{n} \right) \\ &=1-\mathbb P\left( \max X_i \leq \theta_0 - \frac{t}{n} \right) \\ \end{align*}

Since $$\max X_i \leq \theta_0 - \frac{t}{n} \iff \forall i, X_i \leq \theta_0 - \frac{t}{n}$$ and since all $$X_i$$ are independent and identically distributed, \begin{align*} \mathbb P\left( \max X_i \leq \theta_0 - \frac{t}{n} \right) &=\prod_{i=1}^n\mathbb P\left( X_i \leq \theta_0 - \frac{t}{n} \right) \\ &=\mathbb P\left( X_1 \leq \theta_0 - \frac{t}{n} \right)^n \end{align*}

For $$n$$ large enough we have $$\frac{t}{n} \leq \theta_0$$ and then, \begin{align*} \mathbb P\left( X_1 \leq \theta_0 - \frac{t}{n} \right)^n &= \left(\frac{\theta_0 - \frac{t}{n} }{\theta_0} \right)^n \\ &=\left(1 - \frac{t}{\theta_0 n}\right)^n \end{align*}

As $$n \to \infty$$ the last line is equivalent to $$\exp\left(-\frac{t}{\theta_0}\right).$$

So for $$t \geq 0$$: $$\mathbb P\left( n(\theta_0 -\hat \theta) \leq t \right) \xrightarrow[n \to \infty]{} 1- \exp\left(-\frac{t}{\theta_0}\right).$$

We can see that $$1-\exp\left(-\frac{t}{\theta_0}\right)$$ is the cumulative distribution function of an exponential random variable with rate $$\frac{1}{\theta_0}$$.

Thus, $$n (\theta_0 - \hat \theta )$$ is asymptotically exponentially distributed, and the rate of convergence of $$\hat \theta$$ is $$n^{-1}$$.

Here is a R code to illustrate this result (with $$\theta_0 = 3$$ and $$n=50$$):

theta0<-3
n<-50
s<-sapply(1:1e4,function(i){
X<-runif(n,0,theta0)
return(n*(theta0-max(X)))
})
P_emp<-function(t){
return(mean(s<=t))
}
#P_emp is an empirical cumulative distribution of n*(theta0 - MLE)
#P_emp should be close to the CDF of an exponential distribution

x<-seq(0,20,length.out = 1000)
y1<-sapply(x,function(t) P_emp(t))

plot(x,y1,type='l',col="red")
lines(x,pexp(x,rate=1/theta0),type='l',col="black")
legend(x=0,y=1,legend = c("Empirical CDF","Exponential
CDF"),lty=1,col=c("red","black"))

• Thanks! This is completely solved my problem! If I change the distribution to other distributions on $[0,\theta_0]$ and want to estimate the boundary ($\theta_0$) using $\underset{i}{max}\{X_i\}$, does the rate $n^{-1}$ still hold? – T34driver Jul 11 '20 at 6:40
• @JTS365 No I don't think it works for any distribution over $[0, \theta_0]$. Take for example $\theta_0 =1$ and the distribution with density $f(x) = 2-2x$ over $[0,1]$, then $\forall t$, $\mathbb P( n(1- \max X_i) \leq t) \to 0$ which means that $n(1-\max X_i) \to \infty$. In that case the rate is $n^{-\frac{1}{2}}$ – winperikle Jul 11 '20 at 14:34
• Thanks! Does the result generalize to bivariate uniform distribution on a rectangle $[0,\theta_0]\times [0,\theta_1]$? – T34driver Jul 16 '20 at 23:41
• @JTS365 Yes it does, if $(X_0,X_1) \sim U_{[0,\theta_0]\times [0,\theta1]}$ then $X_0 \perp X_1$ and $X_i \sim \mathcal U_{[0,\theta_i]}$ thus $\hat \theta_0 = \max X_{0,i}$ and $\hat \theta_1 = \max X_{1,i}$ and the rate for both estimators is $n^{-1}$. – winperikle Jul 17 '20 at 22:05