I have a process which consists of a number of events and what is known is the timings between the events. What I'm trying to determine is a distribution that allows me to determine a likelyhood that a new sample fits the distibution.

The issue is mainly that if you have lots of samples you can approximate the result using a standard gaussian and use the mean and standard deviation. But if you only have a handful of samples, the gaussian does not accurately represent the situation.

From what I've read it is common to model waiting times using the gamma distribution. Looking at how the process evolves it looks like it matches well. The unknown is the scale parameter, since the shape parameter I think should be the number of samples. What I've worked out so far is that given the timings $X_1 ... X_N$ you can say:

$$ \sum_{n=1}^N X_n \sim \Gamma(N,\theta) $$

($N$ is known and fixed)

However, $\theta$ is unknown, but the maximum likelihood parameter is the average of the $X_i$ (according to wikipedia anyway).

My question is, can I use this to estimate a distribution for $X_i$, that is, since the $X_i$ are independent:

$$ N X_i | \sum_n X_n \sim \Gamma(N, \tfrac{1}{N}\sum_n X_n) $$

Something else I've wondered about. Suppose I do have information about $\theta$, say a distribution. How can I incorporate this into the model?

Edit: Clarified that N is fixed.

  • $\begingroup$ What you're describing above in a round-about way is a Poisson process with rate $\theta$. $\endgroup$
    – cardinal
    Commented Mar 28, 2011 at 23:44
  • $\begingroup$ Well, that's true. There is a close relationship the gamma distribution and Poisson processes. But I don't see how that helps me. $\endgroup$
    – kleptog
    Commented Mar 29, 2011 at 6:15

2 Answers 2


If you have a Poisson process, then each $X_i$ has an exponential distribution (and their sum a Gamma distribution).

You can then use Bayesian methods to look at possible values of $\theta$. For example, the conjugate prior for an exponential distribution is also a Gamma distribution, which you may find helpful incorporating your prior information and observed data to get a credible interval for $\theta$.

  • $\begingroup$ Aha! That's a good tip for the conjugate prior. I didn't realise that's what it could be used for. $\endgroup$
    – kleptog
    Commented Mar 29, 2011 at 16:28

The reason that wikipedia says $\sum_{i=1}^{N}X_{i}\sim\Gamma(N,\theta)$ is actually because we assume a poisson process. That is, the waiting time for the next event is exponentially distributed $X_{i}\sim EXP(\theta)$. You can prove with the moment generating function that given $X_{i}\sim EXP(\theta)$ for $i=1...N$ IID sample, then $\sum_{i=1}^{N}X_{i}\sim\Gamma(N,\theta)$.

Briefly put, if you assumed the $\sum_{i=1}^{N}X_{i}\sim\Gamma(N,\theta)$ model for your data, then you can also assume $X_{i}\sim EXP(\theta)$. (Meaning that's what they assumed originally...)

One last thing, when you write $NX_{i}|\sum_{i=1}^{N}X_{i}\sim\Gamma(N,\frac{1}{N}\sum_{i=1}^{N}X_{i})$, that is not correct. Instead, $NX_{i}|\sum_{i=1}^{N}X_{i}\sim UNIFORM(0,\sum_{i=1}^{N}X_{i})$. To more or less convince yourself, think that given the value of $\sum_{i=1}^{N}X_{i}=M$, then $NX_{i}$ cannot possibly be greater than $M$.

I hope this helps!

EDIT: I'm wrong about the uniform. See comments.

  • $\begingroup$ Why are you mixing $n$ and $N$? The OP appears to be using $N$ as a fixed, known integer. Also neither $N X_i$ nor $X_i$ are uniformly distributed on $[0,M]$ conditional on the given sum. $\endgroup$
    – cardinal
    Commented Mar 29, 2011 at 15:13
  • $\begingroup$ Woops. I wasn't used to typing $N$. I'll change that. However I do believe $X_i$ conditional on the sum is uniform $[0,M/N]$ where $M$ is the sum. $\endgroup$
    – djma
    Commented Mar 29, 2011 at 16:26
  • $\begingroup$ No, the distribution is not uniform. I think you've likely misunderstood the order-statistic property of the Poisson process. A quick way to check is that $X_i$ definitely has positive probability outside that stated interval for each $i$ conditional on the total sum. You can also check the conditional mean via linearity and see that they don't match. $\endgroup$
    – cardinal
    Commented Mar 29, 2011 at 17:28
  • $\begingroup$ Indeed, I don't understand how it can be uniform. The distribution of $X_i$ is indeed exponential. But if you condition it on the sum it has to change, but I don't see how the result can be uniform. $\endgroup$
    – kleptog
    Commented Mar 29, 2011 at 19:20
  • $\begingroup$ The conditional distribution of each $X_i$ conditioned on the total sum is the same as the distribution of the minimum of $N-1$ i.i.d. uniform $[0,M]$ random variables and so has distribution function $1-(1-x/M)^{N-1}$ on $[0,M]$. Note that this distribution has mean $M/N$, as you should expect. There is a close connection to the beta distribution. $\endgroup$
    – cardinal
    Commented Mar 29, 2011 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.