I have some real world data that reflects waiting time in a system. As it's about waiting times I assume it's Gamma distributed and visual check (histogram overlaid by a fitted Gamma PDF) shows no contradiction (except for the cases when failures kick in, but that's fine).

The waiting times in the system have a lower bound (i.e. it's technically not possible to achieve connection time under some bound).

(Q1) Assuming the lower bound is known, should it be subtracted from the data to most properly fit Gamma distribution to it?

(Q2) If the lower bound is known "partially" (at least c), should I subtract the lower estimate of it from the data?

I'm asking these questions, because it feels a bit wrong to see non-zero PDF in the "impossible" range (closer to zero). At the same time I can just admit that this is the chosen model, it does not account for the lower bound, but otherwise works good/acceptable, so this is a trade off. (Q3) Would this be an adequate position as well?

Thank you for sharing your knowledge!

  • 3
    $\begingroup$ You (should) know more about your system than we do, but in loosely similar situations I would lean to fitting a three-parameter version of the distribution. It's hard to imagine in many cases that the minimum response time will ever be known exactly. If you make that assumption you could be embarrassed by a later dataset which shows a smaller (estimate of the) minimum. $\endgroup$ – Nick Cox Apr 30 '19 at 16:32
  • $\begingroup$ You can try running the data through my online statistical distribution fitter at zunzun.com/StatisticalDistributions/1 and see if it might suggest any useful candidate statistical distributions. $\endgroup$ – James Phillips Apr 30 '19 at 17:13
  • 1
    $\begingroup$ I suppose (roughly) gamma data arise from summing components of waiting time modeled as "exponential." [Strictly speaking, sum of exponentials is gamma only if indep & of same rate.] Exponential dist'n is often used to model waiting times because of its simple math form and because 'no-memory' property allows ignoring past history, but they are seldom exactly correct. You may do more harm than good by introducing an additional (possibly squishy) assumption of min waiting times into the model. // Also, note that for shape param's above 5 or so, gamma dist'ns have low probability left tails. $\endgroup$ – BruceET Apr 30 '19 at 18:33

Suppose the total waiting time is the sum of ten exponential components, each with mean $\mu = 5.$ Here are two simple ways to make an approximate model that accounts for a minimum waiting time of $2$ units for each component.

In the R simulation below, $W_1$ adds $2$ to each component (making components non-exponential). By contrast, $W_2$ increases the mean of each exponential component by $2.$

  • Means are equal: $E(W_1) = E(W_2) = 70.$

  • Probabilities of forbidden overall waiting times are similar: $P(W_1 < 20) = 0,$ exactly, $P(W_2 < 20) < 0.001.$ [But I wonder if it is realistic to have $P(W_1 < 30) \approx 0.]$

  • The variance of $W_2$ is larger. [I wonder which variance is more realistic.]

Histograms of $W_1$ (shifted gamma) and $W_2$ are as follows (R code after):

enter image description here

w1 = replicate(10^5, sum(rexp(10, 1/5) + 2)) 
w2 = replicate(10^5, sum(rexp(10, 1/(5+2))))

mean(w1);  sd(w1);  mean(w1 < 20);  mean(w1 < 30)
[1] 70.01318  # aprx E(W1) = 70
[1] 15.77648  # aprx E(W1) = 15.81
[1] 0         # P(W1 < 20) = 0
[1] 2e-05     # P(W1 < 30) aprx 0. Really?

mean(w2);  sd(w2);  mean(w2 < 20)
[1] 70.10161
[1] 22.18051
[1] 0.00074
[1] 0.01289

par(mfrow=c(2,1)); mx = max(w1,w2)
 hist(w1, prob=T, col="skyblue2", xlim=c(0,mx))
  curve(dgamma(x-20, 10, 1/5), add=T, lwd=2)
  abline(v = 20, col="red")
 hist(w2, prob=T, col="skyblue2", xlim=c(0,mx))
  curve(dgamma(x, 10, 1/7), add=T, lwd=2)
| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.