Find the joint distribution of $X_1$ and $\sum_{i=1}^n X_i$

Question

This question is from Robert Hogg's Introduction to Mathematical Statistics 6th version question 7.6.7. The problem is :

Let a random sample of size $n$ be taken from a distribution with the pdf $$f(x;\theta)=(1/\theta)\exp(-x/\theta)\mathbb{I}_{(0,\infty)}(x)$$

Find the MLE and the MVUE of $P(X \le 2)$.

I know how to find the MLE.

I think the idea to find the MVUE is to use Rao-Blackwell and Lehmann and Scheffe. First we find an unbiased estimator of $P(X \le 2)$ which can be $\mathbb{I}_{(0,2)}(X_1)$, and we know $Y=\sum_{i=1}^n X_i$ a sufficient statistic.

Then $\mathbb{E}[I_{(0,2)}(X_1)\mid Y]$ will be the MUVE.

To find the expectation, we need to the joint distribution of $X_1$ and $Y=\sum_{i=1}^n X_i$

I am stuck here.

The book has a solution, but I don't understand the solution. The solution says let us find the joint distribution of $Z=X_1$ and $Y$ but first letting $V=X_1+X_2$ and $U=X_1+X_2+X_3+...$ the Jacobian is one then we integrating out those other variables.

How comes the Jacobian is equal to one?

The answer for the joint distribution is $$g(z,y;\theta)=\frac{(y-z)^{n-2}}{(n-2)!\theta^n}e^{-y/\theta}$$

How do we get this?

Update: As suggested by Xi'an(the book suggested transformation is confusing), let us do the transformation by the following way:

Let

\begin{align} Y_1 & =X_1, \\Y_2 & =X_1+X_2,\\ Y_3 & =X_1+X_2+X_3, \\Y_4 & =X_1+X_2+X_3+X_4, \\ & \quad \vdots \\Y_n & =X_1+X_2+X_3+X_4+\cdots+X_n \end{align}

then

\begin{align} X_1 & =Y_1, \\ X_2 & =Y_2-Y_1,\\ X_3 & =Y_3-Y_2,\\X_4 & =Y_4-Y_3,\\ & \,\,\,\vdots \\ X_n & =Y_n-Y_{n-1} \end{align}

and the corresponding Jacobian is:

$$\left | J \right |=\begin{vmatrix} \frac{\partial x_1}{\partial y_1} &\frac{\partial x_1}{\partial y_2} &\frac{\partial x_1}{\partial y_3} &\cdots &\frac{\partial x_1}{\partial y_n} \\ \frac{\partial x_2}{\partial y_1} &\frac{\partial x_2}{\partial y_2} &\frac{\partial x_2}{\partial y_3} &\cdots &\frac{\partial x_2}{\partial y_n} \\ \frac{\partial x_3}{\partial y_1} &\frac{\partial x_3}{\partial y_2} &\frac{\partial x_3}{\partial y_3} &\cdots &\frac{\partial x_3}{\partial y_n} \\ \vdots&\vdots & \vdots & &\vdots \\ \frac{\partial x_n}{\partial y_1} &\frac{\partial x_n}{\partial y_2} &\frac{\partial x_n}{\partial y_3} &\cdots &\frac{\partial x_n}{\partial y_n} \end{vmatrix}=\begin{array}{r} 1& 0 &0 & \cdots &0 &0 \\ -1& 1 & 0 & \cdots & 0&0\\ 0&-1 & 1 & \cdots &0 &0\\ \vdots & \vdots & \vdots & & \vdots & \vdots \\ 0&0 &0 & \cdots & -1&1 \end{array}=1$$

Since $X_1,X_2,\ldots,X_n$ are i.i.d $\Gamma(1,\theta)$ [or $\mathcal{E}(1/\theta)$], the joint density of $x_1,x_2,\ldots,x_n $ is :

$$f(x_1,x_2,\ldots,x_n)=\frac{1}{\theta}\exp(-x_1/\theta) \times\frac{1}{\theta} \exp(-x_2/\theta)\times \cdots\times\frac{1}{\theta}\exp(-x_n/\theta) \mathbb{I}_{x_1\ge 0} \cdots\mathbb{I}_{x_n\ge 0}$$

Therefore, the joint pdf of $(Y_1,Y_2,\ldots, Y_n)$ is

\begin{align*}h(y_1,y_2,\ldots,y_n)&=\frac{1}{\theta^n}\exp(-y_1/\theta)\exp[-(y_2-y_1)/\theta]\exp[-(y_3-y_2)/\theta]\cdots\exp[-(y_n-y_{n-1})/\theta]\left |J \right |\mathbb{I}_{y_1\ge 0}\mathbb{I}_{y_2-y_1\ge 0}\cdots\mathbb{I}_{y_n-y_{n-1} \ge 0}\\&=\frac{1}{\theta^n}\exp(-y_n/\theta)\mathbb{I}_{y_1\ge 0} \mathbb{I}_{y_2\ge y_1}\cdots\mathbb{I}_{y_n\ge y_{n-1}}\end{align*}

Next, we can integrate out $y_2,y_3,\ldots,y_{n-1}$ to get the joint pdf $y_1$ and $y_n$

Thanks to suggestions from Xi'an, now I can solve the problem, I will give detailed calculations below

\begin{align} g(y_1,y_n) = {} &\int_{y_1}^{y_n}\int_{y_2}^{y_n} \cdots \int_{y_{n-3}}^{y_n} \int_{y_{n-2}}^{y_n} \frac{1}{\theta^n}\exp(-y_n/\theta)dy_{n-1}dy_{n-2} \cdots dy_3\,dy_2\\ = {} & \frac{1}{\theta^n}\exp(-y_n/\theta)\\ & \int_{y_1}^{y_n}\int_{y_2}^{y_n}\cdots\int_{y_{n-3}}^{y_n}\int_{y_{n-2}}^{y_n} \, dy_{n-1}\,dy_{n-2}\cdots dy_3\,dy_2 \\ = {} & \frac{1}{\theta^n}\exp(-y_n/\theta) \int_{y_1}^{y_n}\int_{y_2}^{y_n}\cdots\int_{y_{n-4}}^{y_n}\int_{y_{n-3}}^{y_n}(y_n-y_{n-2})\,dy_{n-2}\,dy_{n-3}\cdots dy_3\,dy_2 \\ = {} & \frac{1}{\theta^n}\exp(-y_n/\theta) \int_{y_1}^{y_n} \int_{y_2}^{y_n} \cdots \int_{y_{n-5}}^{y_n}\int_{y_{n-4}}^{y_n}\frac{(y_n-y_{n-3})^2}{2}dy_{n-3} \,dy_{n-4}\cdots dy_3 \, dy_2 \\ = {} & \frac{1}{\theta^n} \exp(-y_n/\theta) \int_{y_1}^{y_n }\int_{y_2}^{y_n} \cdots \int_{y_{n-6}}^{y_n} \int_{y_{n-5}}^{y_n} \frac{(y_n-y_{n-4})^3}{2 \times 3} \, dy_{n-4} \, dy_{n-5} \cdots dy_3\,dy_2\\ = {} & \frac{1}{\theta^n} \exp(-y_n/\theta) \int_{y_1}^{y_n} \int_{y_2}^{y_n} \cdots \int_{y_{n-7}}^{y_n} \int_{y_{n-6}}^{y_n} \frac{(y_n-y_{n-5})^4}{2 \times 3 \times 4} \, dy_{n-5} \, dy_{n-4} \cdots dy_3\,dy_2\\ = {} & \cdots \\ = {} & \frac{1}{\theta^n}\exp(-y_n/\theta)\frac{(y_n-y_1)^{n-2}}{(n-2)!} \end{align}

Change to the book's notation, $y=y_n, z=y_1$, we get

$$g(z,y;\theta)=\frac{(y-z)^{n-2}}{\theta^n(n-2)!}e^{-y/\theta}.$$

This solves the problem.

Answer 1

The transformation argument works fine and is always useful but I will now suggest an alternative way to solve this problem that bears certain resemblance to the method you would use if the variables were discrete. Recall that the main difference is that while for a discrete random variable $X$ $P(X=x)$ is well-defined for a continuous rv $Y$, $P(Y=y)=0$, so we need to be a little careful.

Let $S=\sum_{i=1}^n X_i$ and we are now looking for the joint distribution

$$f_{X_1, S} \left(x_1, s \right)$$

which we can approximate with the probability

\begin{align} f_{X_1, S} \left(x_1, s \right) &\approx P\left[ x_1 <X_1< x_1 +\Delta x_1 , s<S<s+\Delta s \right] \\ &\approx P\left[ x_1 <X_1< x_1 +\Delta x_1 , s-x_1<\sum_{i=2}^n X_i<s-x_1+\Delta s \right] \\ &= P\left[ x_1 <X_1< x_1 +\Delta x_1 \right] P \left[ s-x_1<\sum_{i=2}^n X_i<s-x_1+\Delta s \right] \\ &\approx \frac{1}{\theta} \exp\left\{-\frac{x_1}{\theta}\right\} \frac{ \left(s-x_1\right)^{n-2} \exp\left\{-\frac{s-x_1}{\theta} \right\}}{\Gamma \left(n-1 \right) \theta^{n-1}} \\ &= \frac{\left(s-x_1\right)^{n-2}}{\theta^n \left(n-2\right)!} \exp\left\{-\frac{s}{\theta} \right\} \end{align}

for $ 0<x_1<s<\infty $. Note that in the fourth line we have used the additivity property of the gamma distribution, of which the exponential is a special case.

If you adjust the notation we are getting the same thing here as above. This method allows you to get away with the multiple integration and that's why I prefer it. Again, be careful in how you define the densities, however.

Hope this helps.

Answer 2

Correct me if I am wrong, but I don't think one needs to find the conditional distribution to find the conditional expectation for the UMVUE. We can find the conditional mean using well-known relations between independent Beta and Gamma variables. Specifically, the fact that if $U$ and $V$ are independent Gamma variates, then $U+V$ is a Gamma variate, and it is independent of the Beta variate $\frac{U}{U+V}$.

Here, note that $X_1\sim\text{Gamma}(1,\frac{1}{\theta})$ and $\sum_{i=2}^nX_i\sim\text{Gamma}(n-1,\frac{1}{\theta})$ are independently distributed. And $X_1+\sum_{i=2}^nX_i\sim\text{Gamma}(n,\frac{1}{\theta})$ is distributed independently of $\dfrac{X_1}{X_1+\sum_{i=2}^nX_i}\sim\text{Beta}(1,n-1)$.

Define $h(X_1,\cdots,X_n)=\begin{cases}1&,\text{ if }X_1\le2\\0&,\text{ otherwise }\end{cases}$

$T=\sum_{i=1}^n X_i$ is complete sufficient for the family of distributions $\{1-\exp(-\frac{x}{\theta}):\theta>0\}$.

So UMVUE of $P(X\le 2)$ is $E(h\mid T)$ by the Lehmann-Scheffe theorem.

We have, \begin{align}E(h\mid T=t)&=P(X_1\le2\mid \sum_{i=1}^n X_i=t)\\&=P\left(\frac{X_1}{\sum_{i=1}^nX_i}\le\frac{2}{t}\mid\sum_{i=1}^n X_i=t\right)\\&=P\left(\frac{X_1}{X_1+\sum_{i=2}^nX_i}\le\frac{2}{t}\mid\sum_{i=1}^n X_i=t\right)\\&=P\left(\frac{X_1}{X_1+\sum_{i=2}^nX_i}\le\frac{2}{t}\right)\\&=\int_0^{2/t}\frac{(1-x)^{n-2}}{B(1,n-1)}\,\mathrm{d}x\\&=1-\left(1-\frac{2}{t}\right)^{n-1}\end{align}

Hence the UMVUE of $P(X\le2)$ should be $1-\left(1-\dfrac{2}{\sum_{i=1}^nX_i}\right)^{n-1}$.