Relationship between poisson and exponential distribution

Question

The waiting times for poisson distribution is an exponential distribution with parameter lambda. But I don't understand it. Poisson models the number of arrivals per unit of time for example. How is this related to exponential distribution? Lets say probability of k arrivals in a unit of time is P(k) (modeled by poisson) and probability of k+1 is P(k+1), how does exponential distribution model the waiting time between them?

Answer 1

I will use the following notation to be as consistent as possible with the wiki (in case you want to go back and forth between my answer and the wiki definitions for the poisson and exponential.)

$N_t$: the number of arrivals during time period $t$

$X_t$: the time it takes for one additional arrival to arrive assuming that someone arrived at time $t$

By definition, the following conditions are equivalent:

$ (X_t > x) \equiv (N_t = N_{t+x})$

The event on the left captures the event that no one has arrived in the time interval $[t,t+x]$ which implies that our count of the number of arrivals at time $t+x$ is identical to the count at time $t$ which is the event on the right.

By the complement rule, we also have:

$P(X_t \le x) = 1 - P(X_t > x)$

Using the equivalence of the two events that we described above, we can re-write the above as:

$P(X_t \le x) = 1 - P(N_{t+x} - N_t = 0)$

But,

$P(N_{t+x} - N_t = 0) = P(N_x = 0)$

Using the poisson pmf the above where $\lambda$ is the average number of arrivals per time unit and $x$ a quantity of time units, simplifies to:

$P(N_{t+x} - N_t = 0) = \frac{(\lambda x)^0}{0!}e^{-\lambda x}$

i.e.

$P(N_{t+x} - N_t = 0) = e^{-\lambda x}$

Substituting in our original eqn, we have:

$P(X_t \le x) = 1 - e^{-\lambda x}$

The above is the cdf of a exponential pdf.

Answer 2

For a Poisson process, hits occur at random independent of the past, but with a known long term average rate $\lambda$ of hits per time unit. The Poisson distribution would let us find the probability of getting some particular number of hits.

Now, instead of looking at the number of hits, we look at the random variable $L$ (for Lifetime), the time you have to wait for the first hit.

The probability that the waiting time is more than a given time value is $P(L \gt t) = P(\text{no hits in time t})=\frac{\Lambda^0e^{-\Lambda}}{0!}=e^{-\lambda t}$ (by the Poisson distribution, where $\Lambda = \lambda t$).

$P(L \le t) = 1 - e^{-\lambda t}$ (the cumulative distribution function). We can get the density function by taking the derivative of this:

$$f(t) = \begin{cases} \lambda e^{-\lambda t} & \mbox{for } t \ge 0 \\ 0 & \mbox{for } t \lt 0 \end{cases}$$

Any random variable that has a density function like this is said to be exponentially distributed.

Answer 3

The other answers do a good job of explaining the math. I think it helps to consider a physical example. When I think about a Poisson process, I always come back to the idea of cars passing on a road. Lambda is the average number of cars that pass per unit of time, let's say 60/hour (lambda = 60). We know, however, that the actual number will vary - some days more, some days less. The Poisson Distribution allows us to model this variability.

Now, an average of 60 cars per hour equates to an average of 1 car passing by each minute. Again though, we know there's going to be variability in the amount of time between arrivals: Sometimes more than 1 minute; other times less. The Exponential Distribution allows us to model this variability.

All that being said, cars passing by on a road won't always follow a Poisson Process. If there's a traffic signal just around the corner, for example, arrivals are going to be bunched up instead of steady. On an open highway, a slow tractor-trailer may hold up a long line of cars, again causing bunching. In these cases, the Poisson Distribution may still work okay for longer time periods, but the exponential will fail badly in modeling arrival times.

Note also that there is huge variability based on time of day: busier during commuting times; much slower at 3am. Make sure that your lambda is reflective of the specific time period you are considering.

Answer 4

The Poisson Distribution is normally derived from the Binomial Distribution (both discrete). This you'll find on Wiki.

However, the Poisson distribution (discrete) can also be derived from the Exponential Distribution (continuous).

I've added the proof to Wiki (link below):

https://en.wikipedia.org/wiki/Talk:Poisson_distribution/Archive_1#Derivation_of_the_Poisson_Distribution_from_the_Exponential_Distribution