The question is from a problem I am trying to solve in Robert Hogg's introduction to Mathematical Statistics 6th version problem 7.2.9 in page 380.

The problem is:

We consider a random sample $X_1, X_2,\ldots ,X_n$ from a distribution with pdf $f(x;\theta)=(1/\theta$)exp($-x/\theta$), $0<x<\infty$. Possibly, in a life testing situation, however, we only observe the first r order statistics $Y_1<Y_2<\cdots <Y_r$.

(a) Record the joint pdf of these order statistics and denote it by $L(\theta)$

(b) Under these conditions, find the mle, $\hat{\theta}$, by maximizing $L(\theta)$.

(c)Find the mgf and pdf of $\hat{\theta}$.

(d) With a slight extension of the definition of sufficiency, is $\hat{\theta}$ a sufficient statistic?

I can solve (a) and (b) but I am completely stuck by (c) therefore cannot forward to (d)

Solve (a):

We know joint pdf for $Y_1,Y_2,\ldots,Y_n$ is $g(y_1,y_2,\ldots,y_n)=n!f(y_1)f(y_2)\cdots f(y_n)$ we just integrate out the (r+1) to n terms we will get joint pfd for $Y_1,Y_2,\ldots,Y_r$.

$h(y_1,y_2,\ldots,y_r)=n!f(y_1)f(y_2)\cdots f(y_r)\int_{n-1}^{\infty} \int_{n-2}^{\infty}\cdots \int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})\cdots f(y_{n-1})f(y_n)dy_{r+1}dy_{r+2}\cdots dy_{n-1}dy_{n}$

$=n!f(y_1)f(y_2)\cdots f(y_r) \int_{n-2}^{\infty}\cdots \int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})\cdots f(y_{n-1})[1-F(y_{n-1})]dy_{r+1}dy_{r+2}\cdots dy_{n-1}$

$=n!f(y_1)f(y_2)\cdots f(y_r)\int_{n-2}^{\infty}\cdots \int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})\cdots f(y_{n-2})(-1)[1-F(y_{n-1})]dy_{r+1}dy_{r+2}\cdots dy_{n-2}]d[1-F(y_{n-1})]$

$=n!f(y_1)f(y_2)\cdots f(y_r)\int_{n-2}^{\infty}\cdots \int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})\cdots f(y_{n-2})\frac{[1-F(y_{n-2})]^2}{2}dy_{r+1}dy_{r+2}\cdots dy_{n-2}$

$=n!f(y_1)f(y_2)\cdots f(y_r)\frac{[1-F(y_r)]^{n-r}}{(n-r)!}$

$=n!\frac{1}{\theta}e^{\frac{-y_1}{\theta}}\frac{1}{\theta}e^{\frac{-y_2}{\theta}}\cdots \frac{1}{\theta}e^{\frac{-y_r}{\theta}}[e^{-y_r/\theta}]^{n-r}/(n-r)!$



This part is not difficult. It just a normal way to calculate mle.

$\log L(\theta;y)=\log \frac{n!}{(n-2)!}-r\log(\theta)-\frac{1}{\theta}[\sum_{i=1}^{r}y_i+(n-r)y_r]$ Take derivative of the log likelihood function we get: $\partial \frac{L(\theta;y)}{\theta}=\frac{1}{\theta^2}[\sum_{i=1}^{r}y_i+(n-r)y_r]-r\frac{1}{\theta}$

Set the derivative to zero

We get: $\hat{\theta}=\frac{[\sum_{i=1}^{r}y_i+(n-r)y_r]}{r}$


To solve (c) I think we need at least to know the distribution of $\sum_{i=1}^{r}y_i$.

I search the internet, there is a paper talk about this distribution, https://www.ocf.berkeley.edu/~wwu/articles/orderStatSum.pdf

But I think the method might not be correct since for order statistic $F(y_i)$ are different, we cannot use binomial distribution there.

There is another paper here http://www.jstor.org/stable/4615746?seq=1#page_scan_tab_contents

But I am totally lost at formula (2.2) if someone would like to explain the paper with more detailed calculations, it will be highly appreciated.

(d) only after solve (c)

  • 1
    $\begingroup$ I think a transformation argument will work. $\endgroup$
    – JohnK
    Commented Jul 13, 2015 at 16:50
  • $\begingroup$ Would like to give more explanations? Thanks $\endgroup$
    – Deep North
    Commented Jul 13, 2015 at 23:30
  • 1
    $\begingroup$ @DeepNorth: in (a), all your integrals have a wrong lower bound. $\endgroup$
    – Xi'an
    Commented Aug 1, 2015 at 17:17
  • $\begingroup$ Thanks, I am also wondering if I am right for the bound. $\endgroup$
    – Deep North
    Commented Aug 2, 2015 at 1:29
  • $\begingroup$ In the first equation,$$\int_{n-1}^{\infty} \int_{n-2}^{\infty}...\int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})...$$should be$$\int_{y_{n-1}}^{\infty} \int_{y_{n-2}}^{\infty}...\int_{y_{r+1}}^{\infty} \int_{y_{r}}^{\infty}f(y_{r+1})f(y_{r+2})...$$and so on... $\endgroup$
    – Xi'an
    Commented Aug 2, 2015 at 8:06

2 Answers 2


Since$$(y_1,\ldots,y_r)\sim\frac{n!\theta^{-r}}{(n-r)!}e^{-\frac{1}{\theta}[\sum_{i=1}^{r}y_i+(n-r)y_r]}\mathbb{I}_{y_\le y_2\le \ldots \le y_r}$$you have the joint pdf of $(y_1,\ldots,y_r)$. From there, you can deduce the pdf of $$s_r=\sum_{i=1}^{r}y_i+(n-r)y_r\,.$$Indeed, because the Jacobian of the transform is constant,\begin{align*}f_s(y_1,\ldots,y_{r-1},s_r) &\propto f_Y\left(y_1,\ldots,\left\{s_r-\sum_{i=1}^{r-1}y_i\right\}\Big/(n-r+1)\right) \\&\propto \theta^{-r} \exp\{-s_r/\theta\}\mathbb{I}_{y_\le y_2\le \ldots \le\left\{s_r-\sum_{i=1}^{r-1}y_i\right\}/(n-r+1)}\end{align*}implies by integration in $y_1,\ldots,y_{r-1}$ that$$f_s(s_r)\propto\theta^{-r} \exp\{-s_r/\theta\}s_r^{r-1}$$ Indeed, \begin{align*} f_s(s_r)&=\int\cdots\int f_s(y_1,\ldots,y_{r-1},s_r)\text{d}y_1\cdots\text{d}y_{r-1}\\ &= \theta^{-r} \exp\{-s_r/\theta\}\int\cdots\int \mathbb{I}_{y_\le y_2\le \ldots \le\left\{s_r-\sum_{i=1}^{r-1}y_i\right\}/(n-r+1)}\text{d}y_1\cdots\text{d}y_{r-1} \end{align*} leads to constraint $y_{r-1}$ by $y_{r-2}\le y_{r-1}$ and by $$y_{r-1}\le \left\{s_r-\sum_{i=1}^{r-1}y_i\right\}/(n-r+1)=\left\{s_r-\sum_{i=1}^{r-2}y_i\right\}/(n-r+1)-\frac{y_{r-1}}{n-r+1}$$ which simplifies into $$y_{r-1}\le \left\{s_r-\sum_{i=1}^{r-2}y_i\right\}/(n-r+2)$$ If one starts integrating in $y_{r-1}$, the most inner integral is \begin{align*}\int_{y_{r-2}}^{\{s_r-\sum_{i=1}^{r-2}y_i\}/(n-r+2)}\text{d}y_{r-1}&=\left\{s_r-\sum_{i=1}^{r-2}y_i\right\}/(n-r+2)-y_{r-2}\\ &=\left\{s_r-\sum_{i=1}^{r-3}y_i\right\}/(n-r+2)-\frac{(n-r+1)y_{r-2}}{n-r+2} \end{align*} and from there one can proceed by recursion.


Here is an R simulation to show the fit: enter image description here obtained as follows

  • $\begingroup$ Xi'an, please, can you elaborate on the evaluation of the integral? $\int \cdots \int \mathbb{I}_{y_1, \ldots, y_r} dy_1 \cdots d_yr$. I can write the limits as $\int_0^{\frac{s_r}{2}}dy_1 \int_{y_1}^{\frac{s_r - y_1}{2}}dy_2 \cdots \int_{y_m}^{\frac{s_r - \sum_{j=1}^{m}y_j}{2}}dy_m \cdots \int_{y_{r-1}}^{\frac{s_r - \sum_{j=1}^{r-1}y_j}{2}} dy_r$. I am not able to see how this can be carried out. I have also posted a related question in math overflow. $\endgroup$
    – them
    Commented Aug 25, 2016 at 10:21
  • $\begingroup$ @them: I added some details for your sake. $\endgroup$
    – Xi'an
    Commented Aug 25, 2016 at 21:13

I am also self-studying introduction to Mathematical Statistics 7th edition and I solved the same problem yesterday. This site has helped me a lot. I would like to contribute it too. I am not a native English speaker so please don't mind if my expression is not natural.

The problem asked us to find mgf of $\hat \theta$. We can find it using mgf of $y_1,y_2,...,y_r$ that is $M_{\vec Y_r} ( \vec t)$.

Since $$f_{\vec Y_r}= {n!\over(n-r)! \theta^r} e^{-{1 \over \theta } (\sum^r_{i=1} {y_i}+(n-r)y_r)} $$ and $$M_{\vec Y_r} ( \vec t)=M_{\vec Y_r}((t_1,t_2,...,t_r)')=E[e^{t'Y}] $$ , this is what we get : $$M_{\vec Y_r} ((t_1,t_2,...,t_r)')=\int_A {n!\over(n-r)! \theta^r} e^{ \sum^{r-1}_{i=1} {(t_i-{1 \over \theta })y_i}+(t_r-{(n-r+1)\over \theta})y_r} dy_1 dy_2 ...dy_r $$ where $A=\{(y_1,y_2,...,y_r)| 0<y_1\leq y_2 \leq ...\leq y_r < \infty \}$and $t_i <{1\over\theta}$.

We are interested in $\hat \theta= {1\over r}{(\sum^r_{i=1} {y_i}+(n-r)y_r)} $ which has the mgf, $M_{\hat \theta}(t)=E[e^{t \hat\theta}]$. And this is exactly the same with $M_{\vec Y_r} (({t\over r},{t\over r},...,{t\over r},{{n-r+1}\over r}t)')$

Here is some trick to simplify this, let $q_i=e^{-({1\over \theta}-{t\over r}) y_i}$ where $1 >q_1 \ge q_2 \ge ... \ge q_n >0$ and express the integral with $q_i$ s.

$$M_{\hat \theta}(t)={n!\over(n-r)! \theta^r} {1 \over { ({1 \over \theta }-{t \over r})^r}} \int_0^1 \int_0^{q_1}...\int_0^{q_{r-1}} \int_0^{q_r} q_r^{(n-r)} dq_r dq_{r-1}...dq_2 dq_1 $$

You can calculate this easily.

$$M_{\hat \theta}(t)=(1-{\theta \over r}t)^{-r}$$

So you can conclude $$\hat\theta \sim \Gamma(r,{\theta \over r})$$ which has the mean $\theta$.

Thank you for reading my first answer on stackexchange.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.