Distribution given sum

Question

I'm stuck on an exercise (it's not homework, but preparation for finals). It goes like this: $X_1, \dots, X_n$ are iid Exponential($\lambda$) (with parametrization $f(x)=\lambda e^{-\lambda x}$). What is the pdf $f(x_n|Y)$, where $Y=\sum_{i=1}^n X_i$? I know that $Y\sim Gamma(n, \lambda^{-1})$ with pdf $$ f(y)=\frac{\lambda^n}{\Gamma(n)}y^{n-1}e^{-y\lambda}. $$

So I have the marginals, but I'm not sure how to proceed. Maybe it's easier to get $f(y|X_n)$ and then multiply by the ratio of the marginals? Any help that puts me in the right direction is appreciated!

Edit: I think I solved it. Since all are iid, the joint is simply $f(x_n, t_{n-1})$ where $T_{k}=\sum_{i=1}^kX_i$. This comes from a product of convolutions. The joint of $(x_2, x_1)$ is just the product of the densities. The joint of $$ f(x_3, t_2)=f_{X_3}(x_3)f_{T_2}(t_3-x_3) $$ and so on. So finally $$ f(x_n, t_{n-1})=f_{X_n}(x_n)f_{T_{n-1}}(t_n-x_n). $$ The former is exponential, the latter is gamma, so the conditional is $$ f(x_n|T_n)=\frac{f_{X_n}(x_n)f_{T_{n-1}}(t_n-x_n)}{f_{T_n}(t_n)}. $$

Answer 1

It can be instructional and satisfying to work this out using basic statistical knowledge, rather than just doing the integrals. It turns out that no calculations are needed!

Here's the circle of ideas:

• The $X_i$ can be thought of as waiting times between random events.

• When the waiting times have independent identical exponential distributions, the random events are a Poisson process.

• When normalized by the last time (given by $Y=X_1+X_2+\cdots + X_n$), these events therefore look like $n-1$ independent uniform values in $[0,1]$.

• The values $0 \le X_1/Y \le X_1/Y+X_2/Y \le \cdots \le (X_1/Y+\cdots+X_{n-1}/Y) \le 1$ therefore are the order statistics for $n-1$ iid uniform variables.

• The $k^\text{th}$ order statistic has a Beta$(k, n-k)$ distribution.

• The PDF of a Beta$(k,n-k)$ distribution is proportional to $x^{k-1}(1-x)^{n-k-1}$ for $0\le x \le 1$, with constant of proportionality equal to (of course!) the reciprocal of the Beta function value $B(k,n-k)$.

• Since $Y$ is invariant under any permutation of the $X_i$ and the $X_i$ are exchangeable, all the conditional distributions $f(X_i|Y)$ are the same.

Thus, the distribution of any of the $X_i$ conditional on $Y$ must be $Y$ times a Beta$(1,n-1)$ distribution. Scaling the Beta PDF by $y$ gives the conditional probability element

$$f_{X_1|Y=y}(x)\, \mathrm{d}x = \frac{1}{B(1,n-1)}\left(1-\frac{x}{y}\right)^{n-2}\frac{\mathrm{d}x}{y}$$

for $0 \le X_i \le y$.

This reasoning further implies the $n$-variate distribution of the $X_i$ conditional on $Y$ is $Y$ times a symmetric Dirichlet distribution.

Reference

Balakrishnan, N. and A. Clifford Cohen, Order Statistics and Inference. Academic Press, 1991.

Answer 2

This might be the most "text book" answer on $f_{X_1|Y}(x_1|y)$.

Let $Z = X_2 + ... + X_n$. Then $Y = Z + X_1$.

First the joint distribution of $(Z, X_1)$ which is $f_{Z,X_1}(z,x_1) = \dfrac{\lambda^{n-1}}{\Gamma(n-1)}\,z^{n-2}\,e^{-z\lambda}\,\lambda \times e^{-x_1 \lambda}$ for $z \ge 0, x_1 \ge 0$.

Next we get the joint distribution of $(Y, X_1)$, which is $f_{Y,X_1}(y,x_1) = f_{Z,X_1}(z,x_1)$ since the absolute determinant of the Jacobian matrix is 1 (details below).

Therefore, $f_{Y,X_1}(y,x_1) = \dfrac{\lambda^n}{\Gamma(n-1)}\,(y-x_1)^{n-2}\,e^{-y\lambda}$ for $x_1 \ge 0, y \ge x_1$, and $0$ otherwise.

Finally, $f_{X_1|Y}(x_1|y) = \dfrac{f_{Y,X_1}(y,x_1)}{f_Y(y)}$. The rest follows easily from here.

Edit: I didn't see OP's "edit" part at first. So OP has solved it in the same "text book" way. I will leave my post here, in case it should be useful to others.

Edit 2: Details on the Jacobian.

We are doing variable transformations, from $Z, X_1$ to $Y, X_1$, defined as $Y=Z+X_1, X_1 = X_1$. So Jacobian matrix is 2-by-2, with the elements being (going across rows) $\dfrac{\partial Y}{\partial Z}, \dfrac{\partial Y}{\partial X_1}, \dfrac{\partial X_1}{\partial Z}, \dfrac{\partial X_1}{\partial X_1}$, i.e. $1,1,0,1.$ The absolute determinant of this matrix is $1.$