7
$\begingroup$

The question is from a problem I am trying to solve in Robert Hogg's introduction to Mathematical Statistics 6th version problem 7.2.9 in page 380.

The problem is:

We consider a random sample $X_1, X_2,\ldots ,X_n$ from a distribution with pdf $f(x;\theta)=(1/\theta$)exp($-x/\theta$), $0<x<\infty$. Possibly, in a life testing situation, however, we only observe the first r order statistics $Y_1<Y_2<\cdots <Y_r$.

(a) Record the joint pdf of these order statistics and denote it by $L(\theta)$

(b) Under these conditions, find the mle, $\hat{\theta}$, by maximizing $L(\theta)$.

(c)Find the mgf and pdf of $\hat{\theta}$.

(d) With a slight extension of the definition of sufficiency, is $\hat{\theta}$ a sufficient statistic?

I can solve (a) and (b) but I am completely stuck by (c) therefore cannot forward to (d)

Solve (a):

We know joint pdf for $Y_1,Y_2,\ldots,Y_n$ is $g(y_1,y_2,\ldots,y_n)=n!f(y_1)f(y_2)\cdots f(y_n)$ we just integrate out the (r+1) to n terms we will get joint pfd for $Y_1,Y_2,\ldots,Y_r$.

$h(y_1,y_2,\ldots,y_r)=n!f(y_1)f(y_2)\cdots f(y_r)\int_{n-1}^{\infty} \int_{n-2}^{\infty}\cdots \int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})\cdots f(y_{n-1})f(y_n)dy_{r+1}dy_{r+2}\cdots dy_{n-1}dy_{n}$

$=n!f(y_1)f(y_2)\cdots f(y_r) \int_{n-2}^{\infty}\cdots \int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})\cdots f(y_{n-1})[1-F(y_{n-1})]dy_{r+1}dy_{r+2}\cdots dy_{n-1}$

$=n!f(y_1)f(y_2)\cdots f(y_r)\int_{n-2}^{\infty}\cdots \int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})\cdots f(y_{n-2})(-1)[1-F(y_{n-1})]dy_{r+1}dy_{r+2}\cdots dy_{n-2}]d[1-F(y_{n-1})]$

$=n!f(y_1)f(y_2)\cdots f(y_r)\int_{n-2}^{\infty}\cdots \int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})\cdots f(y_{n-2})\frac{[1-F(y_{n-2})]^2}{2}dy_{r+1}dy_{r+2}\cdots dy_{n-2}$

$=n!f(y_1)f(y_2)\cdots f(y_r)\frac{[1-F(y_r)]^{n-r}}{(n-r)!}$

$=n!\frac{1}{\theta}e^{\frac{-y_1}{\theta}}\frac{1}{\theta}e^{\frac{-y_2}{\theta}}\cdots \frac{1}{\theta}e^{\frac{-y_r}{\theta}}[e^{-y_r/\theta}]^{n-r}/(n-r)!$

$=\frac{n!\theta^{-r}}{(n-r)!}e^{-\frac{1}{\theta}[\sum_{i=1}^{r}y_i+(n-r)y_r]}$

(b)

This part is not difficult. It just a normal way to calculate mle.

$\log L(\theta;y)=\log \frac{n!}{(n-2)!}-r\log(\theta)-\frac{1}{\theta}[\sum_{i=1}^{r}y_i+(n-r)y_r]$ Take derivative of the log likelihood function we get: $\partial \frac{L(\theta;y)}{\theta}=\frac{1}{\theta^2}[\sum_{i=1}^{r}y_i+(n-r)y_r]-r\frac{1}{\theta}$

Set the derivative to zero

We get: $\hat{\theta}=\frac{[\sum_{i=1}^{r}y_i+(n-r)y_r]}{r}$

(c)

To solve (c) I think we need at least to know the distribution of $\sum_{i=1}^{r}y_i$.

I search the internet, there is a paper talk about this distribution, https://www.ocf.berkeley.edu/~wwu/articles/orderStatSum.pdf

But I think the method might not be correct since for order statistic $F(y_i)$ are different, we cannot use binomial distribution there.

There is another paper here http://www.jstor.org/stable/4615746?seq=1#page_scan_tab_contents

But I am totally lost at formula (2.2) if someone would like to explain the paper with more detailed calculations, it will be highly appreciated.

(d) only after solve (c)

$\endgroup$
  • 1
    $\begingroup$ I think a transformation argument will work. $\endgroup$ – JohnK Jul 13 '15 at 16:50
  • $\begingroup$ Would like to give more explanations? Thanks $\endgroup$ – Deep North Jul 13 '15 at 23:30
  • 1
    $\begingroup$ @DeepNorth: in (a), all your integrals have a wrong lower bound. $\endgroup$ – Xi'an Aug 1 '15 at 17:17
  • $\begingroup$ Thanks, I am also wondering if I am right for the bound. $\endgroup$ – Deep North Aug 2 '15 at 1:29
  • $\begingroup$ In the first equation,$$\int_{n-1}^{\infty} \int_{n-2}^{\infty}...\int_{r+1}^{\infty} \int_{r}^{\infty}f(y_{r+1})f(y_{r+2})...$$should be$$\int_{y_{n-1}}^{\infty} \int_{y_{n-2}}^{\infty}...\int_{y_{r+1}}^{\infty} \int_{y_{r}}^{\infty}f(y_{r+1})f(y_{r+2})...$$and so on... $\endgroup$ – Xi'an Aug 2 '15 at 8:06
10
$\begingroup$

Since$$(y_1,\ldots,y_r)\sim\frac{n!\theta^{-r}}{(n-r)!}e^{-\frac{1}{\theta}[\sum_{i=1}^{r}y_i+(n-r)y_r]}\mathbb{I}_{y_\le y_2\le \ldots \le y_r}$$you have the joint pdf of $(y_1,\ldots,y_r)$. From there, you can deduce the pdf of $$s_r=\sum_{i=1}^{r}y_i+(n-r)y_r\,.$$Indeed, because the Jacobian of the transform is constant,\begin{align*}f_s(y_1,\ldots,y_{r-1},s_r) &\propto f_Y\left(y_1,\ldots,\left\{s_r-\sum_{i=1}^{r-1}y_i\right\}\Big/(n-r+1)\right) \\&\propto \theta^{-r} \exp\{-s_r/\theta\}\mathbb{I}_{y_\le y_2\le \ldots \le\left\{s_r-\sum_{i=1}^{r-1}y_i\right\}/(n-r+1)}\end{align*}implies by integration in $y_1,\ldots,y_{r-1}$ that$$f_s(s_r)\propto\theta^{-r} \exp\{-s_r/\theta\}s_r^{r-1}$$ Indeed, \begin{align*} f_s(s_r)&=\int\cdots\int f_s(y_1,\ldots,y_{r-1},s_r)\text{d}y_1\cdots\text{d}y_{r-1}\\ &= \theta^{-r} \exp\{-s_r/\theta\}\int\cdots\int \mathbb{I}_{y_\le y_2\le \ldots \le\left\{s_r-\sum_{i=1}^{r-1}y_i\right\}/(n-r+1)}\text{d}y_1\cdots\text{d}y_{r-1} \end{align*} leads to constraint $y_{r-1}$ by $y_{r-2}\le y_{r-1}$ and by $$y_{r-1}\le \left\{s_r-\sum_{i=1}^{r-1}y_i\right\}/(n-r+1)=\left\{s_r-\sum_{i=1}^{r-2}y_i\right\}/(n-r+1)-\frac{y_{r-1}}{n-r+1}$$ which simplifies into $$y_{r-1}\le \left\{s_r-\sum_{i=1}^{r-2}y_i\right\}/(n-r+2)$$ If one starts integrating in $y_{r-1}$, the most inner integral is \begin{align*}\int_{y_{r-2}}^{\{s_r-\sum_{i=1}^{r-2}y_i\}/(n-r+2)}\text{d}y_{r-1}&=\left\{s_r-\sum_{i=1}^{r-2}y_i\right\}/(n-r+2)-y_{r-2}\\ &=\left\{s_r-\sum_{i=1}^{r-3}y_i\right\}/(n-r+2)-\frac{(n-r+1)y_{r-2}}{n-r+2} \end{align*} and from there one can proceed by recursion.

Hence$$s_r\sim\mathcal{G}a(r,1/\theta)$$

Here is an R simulation to show the fit: enter image description here obtained as follows

n=10
r=5    
sim=matrix(rexp(n*1e4),1e4,n)
sim=t(apply(sim,1,sort))
res=apply(sim[,1:r],1,sum)+(n-r)*sim[,5]
hist(res,prob=TRUE)
curve(dgamma(x,sh=(n-r),sc=1),add=TRUE)
$\endgroup$
  • $\begingroup$ Xi'an, please, can you elaborate on the evaluation of the integral? $\int \cdots \int \mathbb{I}_{y_1, \ldots, y_r} dy_1 \cdots d_yr$. I can write the limits as $\int_0^{\frac{s_r}{2}}dy_1 \int_{y_1}^{\frac{s_r - y_1}{2}}dy_2 \cdots \int_{y_m}^{\frac{s_r - \sum_{j=1}^{m}y_j}{2}}dy_m \cdots \int_{y_{r-1}}^{\frac{s_r - \sum_{j=1}^{r-1}y_j}{2}} dy_r$. I am not able to see how this can be carried out. I have also posted a related question in math overflow. $\endgroup$ – them Aug 25 '16 at 10:21
  • $\begingroup$ @them: I added some details for your sake. $\endgroup$ – Xi'an Aug 25 '16 at 21:13
2
$\begingroup$

I am also self-studying introduction to Mathematical Statistics 7th edition and I solved the same problem yesterday. This site has helped me a lot. I would like to contribute it too. I am not a native English speaker so please don't mind if my expression is not natural.

The problem asked us to find mgf of $\hat \theta$. We can find it using mgf of $y_1,y_2,...,y_r$ that is $M_{\vec Y_r} ( \vec t)$.

Since $$f_{\vec Y_r}= {n!\over(n-r)! \theta^r} e^{-{1 \over \theta } (\sum^r_{i=1} {y_i}+(n-r)y_r)} $$ and $$M_{\vec Y_r} ( \vec t)=M_{\vec Y_r}((t_1,t_2,...,t_r)')=E[e^{t'Y}] $$ , this is what we get : $$M_{\vec Y_r} ((t_1,t_2,...,t_r)')=\int_A {n!\over(n-r)! \theta^r} e^{ \sum^{r-1}_{i=1} {(t_i-{1 \over \theta })y_i}+(t_r-{(n-r+1)\over \theta})y_r} dy_1 dy_2 ...dy_r $$ where $A=\{(y_1,y_2,...,y_r)| 0<y_1\leq y_2 \leq ...\leq y_r < \infty \}$and $t_i <{1\over\theta}$.

We are interested in $\hat \theta= {1\over r}{(\sum^r_{i=1} {y_i}+(n-r)y_r)} $ which has the mgf, $M_{\hat \theta}(t)=E[e^{t \hat\theta}]$. And this is exactly the same with $M_{\vec Y_r} (({t\over r},{t\over r},...,{t\over r},{{n-r+1}\over r}t)')$

Here is some trick to simplify this, let $q_i=e^{-({1\over \theta}-{t\over r}) y_i}$ where $1 >q_1 \ge q_2 \ge ... \ge q_n >0$ and express the integral with $q_i$ s.

$$M_{\hat \theta}(t)={n!\over(n-r)! \theta^r} {1 \over { ({1 \over \theta }-{t \over r})^r}} \int_0^1 \int_0^{q_1}...\int_0^{q_{r-1}} \int_0^{q_r} q_r^{(n-r)} dq_r dq_{r-1}...dq_2 dq_1 $$

You can calculate this easily.

$$M_{\hat \theta}(t)=(1-{\theta \over r}t)^{-r}$$

So you can conclude $$\hat\theta \sim \Gamma(r,{\theta \over r})$$ which has the mean $\theta$.

Thank you for reading my first answer on stackexchange.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.