19
$\begingroup$

I have a servlet-based application wherein I measure the time taken to complete each request to that servlet. I already compute simple statistics like the mean and maximum; I'd like to produce some more sophisticated analysis however, and to do so I believe I need to properly model these response times.

Surely, I say, response times follow some well-known distribution, and there are good reasons to believe that distribution is the right model. However, I don't know what this distribution ought to be.

Log-normal and Gamma come to mind, and you can make either one sort of fit real response time data. Does anyone have a view on what distribution the response times ought to follow?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
21
$\begingroup$

The Log-Normal distribution is the one I find best at describing latencies of server response times across all the user base over a period of time.

A log-normal (or lognormal) distribution is a continuous probability distribution of a random variable whose logarithm is normally distributed. Thus, if the random variable X is log-normally distributed, then: $$Y = ln(X)$$ has a normal distribution.

You may see some examples at the aptly-named site lognormal.com whose in the business of measuring site latency distribution over time and more. I have no affiliation with the site except for being a happy user. Here's how the distribution looks like; response (e.g web page load) time vs number of responses:

a log-normal distribution

Notes:

(1) In this chart, the load-time (X-axis) scale is linear. If you switch the x-axis to a log-scale, the shape of the distribution would look more "normal" on the right side of the peak.

(2) A log normal distribution value is always positive. Obviously, a server can't respond in negative time.

(3) The log normal distribution is right-skewed. The longer tail is on the right of the peak.

(4) Update 2023-08: lognormal.com has been acquired by Soasta, which was later acquired by Akamai, so the original lognormal.com page no longer functions as it used to when the original answer was posted. Instead it redirects to some Akamai page. The example server-response-time distribution (CDF) chart was saved from lognormal.com years ago when the service was still active and remains useful (it is hosted on i.stack.imgur.com)

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ This PDF really looks like a Fréchet in my opinion. $\endgroup$
    – usεr11852
    May 28, 2015 at 18:56
7
$\begingroup$

Example Graph. See article for more depth.

My research shows the best model is determined by a few things: 1) Are you concerned with the body, the tail, or both? If not "both", modeling a filtered dataset can be more useful. 2) Do you want a very simple or a very accurate one? i.e. how many parameters?

If the answer to 1 was "both" and 2 was "simple", Pareto seems to work best. Otherwise, if 1 was "body" and 2 was "simple" - choose a filtered erlang model. If 1 was "both" and 2 was "accurate", you probably want a gaussian mixture model on your data in the log domain - effectively a lognormal fit.

I've been researching this lately, and I didn't find the topic to be covered well enough on the public internet, so I just wrote a blog post detailing my research into this topic.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thanks for the chart. Based on the (roughly) tri-modal distribution you have, I believe this is not a simple (single server) setting. You seem to have some middleware or back-ends that are slower. These cause the overall response to slow down when the user-facing server waits for potentially-cached) back-end subsystems to respond. Also it isn't clear what the X and Y axes represent. Have you inverted the load-time (originally X-axis) and counts (originally Y-axis)? $\endgroup$
    – arielf
    Oct 29, 2015 at 0:27
  • $\begingroup$ Thanks for your feedback! The source dataset was more akin to pings than web service requests, but I would guess the trimodal distribution is due to mainly two things: 1) The main bi-modal asymmetry is due to two network paths, while 2) the long-tail 3rd component is due to tcp error recovery scenarios. That's just a guess though... my main focus was on the empirical utility of various models, not the process and theory. I'm not entirely sure what you are asking about the inverted axis, though... do you have an example plot? $\endgroup$
    – Andrew Charneski
    Oct 29, 2015 at 5:11
  • $\begingroup$ Also, my apologies on the sloppy graphic. The x-axis is microseconds, and the y axis is probability density. (Yeah, I know... sorry... see the notebook for reproducible science.) $\endgroup$
    – Andrew Charneski
    Oct 29, 2015 at 5:18
  • $\begingroup$ The blog post detailing your research into this topic is 404 Not Found $\endgroup$
    – user2579566
    May 28, 2020 at 15:27
  • $\begingroup$ Fixed - I recently replaced what I build my blog on. Thank you! $\endgroup$
    – Andrew Charneski
    May 29, 2020 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.