8
$\begingroup$

I found out that webserver response times are typically modeled as coming from a lognormal distribution here. What I don't quite get is why this is the case!

In particular, Wikipedia states that a random variable X is distributed lognormally when it is the product of several independent normal variables.

In that case, what would these individual normal variables represent in the execution time of server code?

I haven't managed to find any sources that discuss why the lognormal distribution is a good fit for webserver response times.

Not sure if this question would be best asked on stackoverflow or math.stackexchange but thought I'd try here.

Thanks for any insight!

Improve this question
$\endgroup$
4
  • $\begingroup$ Hm... I agree I would think of Fréchet as being more plausible. $\endgroup$
    – usεr11852
    May 28, 2015 at 18:56
  • 3
    $\begingroup$ Lognormals arise in more general situations than as products of independent Normal variates (which is only an approximation anyway): they appear as products of approximately independent values that don't exhibit a huge range among their dispersions. See stats.stackexchange.com/questions/3707 for an example. Thus lognormal distributions are popular models in circumstances where variation may occur through the cumulative action of multiplicative deviations. This is surprisingly common. For instance, it's a basis for Benford's Law. $\endgroup$
    – whuber ♦
    May 28, 2015 at 19:52
  • 2
    $\begingroup$ Re your title: they aren't actually lognormal. That people model them that way does not in any way imply that they're actually distributed lognormally (i.e. you should make your title relate better to your actual question). If you haven't seen good justification for it, you should also entertain the possibility there isn't any. It may also be the case that sometimes it's a good approximation - any paper that says "well, we looked at a bunch of data and it looks pretty good" may justify using it for that data -- it doesn't establish it in general. Perhaps it's often a convenient model. $\endgroup$
    – Glen_b
    May 29, 2015 at 2:10
  • $\begingroup$ Is this restricted to Web (http) servers? Or is that just the server type with the most extensive performance data? HTTP runs over TCP, which uses exponential backoff to handle network congestion. $\endgroup$
    – Livius
    Jun 16, 2015 at 3:58

2 Answers 2

Reset to default
3
$\begingroup$

You might be interested in reading the paper

Vern Paxson. Empirically-Derived Analytic Models of Wide-Area TCP Connections. IEEE/ACM Transactions on Networking, 1994.

which is available online here. From the abstract:

We analyze 3 million TCP connections that occurred during 15 wide-area traffic traces. The traces were gathered at five “stub” networks and two internetwork gateways, providing a diverse look at wide-area traffic. We derive analytic models describing the random variables associated with telnet, nntp, smtp, and ftp connections.

and from the paper

For most connections the responder/duration ratio was well modeled by an exponential distribution, but “large” connections—those whose responder bytes were in the upper 10% of all connections—had a different distribution. For these, the ratio was fairly well modeled by a log-normal distribution.

Though, it is a bit dated already :-)

Improve this answer
$\endgroup$
2
$\begingroup$

Say $X_1, \ldots X_n \overset{iid}{\sim} \text{something}$. You don't know the distribution but you don't need to. By the central limit theorem $\bar{X} \to \mathcal{N}$. Don't worry about the parameters. Then $\exp(\bar{X}) \to \text{lognormal}$. Last part: $Y_1 = \exp(\frac{\sum_i X_i}{n}) = \left[\exp(\sum_i X_i)\right] ^{1/n} = \left[\prod_i \exp X_i\right]^{1/n}$.

$Y_1$ is your first response time. I don't know anything about this stuff, but their justification for using this probably has something to do with this. Probably your response rate comes from an average of a gajillion unknown things coming from some unknown distribution.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Could you be more explicit about how you think this answer responds to the question, "why the lognormal distribution is a good fit for webserver response times"? $\endgroup$
    – whuber ♦
    May 28, 2015 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.