# Statistical test for computer-systems performance analysis?

Suppose we do the following study. We have 2 web-servers (server A and server B) which are processing user requests. Both run on the same hardware, but one of them (server B) uses a slightly different algorithm for request processing. When a user request comes in we have 50/50 chance of processing it on each server. So the work being done by each server is the same in long term.

We have 1 hour system-log containing information about all the requests in this hour (all the population, not sample): what server processed the request and how long it took to process.

For example:

21:00:00 SERVER-A 0.11s
21:00:00 SERVER-A 0.13s
21:00:00 SERVER-B 0.09s
21:00:00 SERVER-A 0.16s
21:00:00 SERVER-B 0.11s
21:00:01 SERVER-A 0.14s


The log is quite large (~250K requests from each server, ~500K in total).

It turns out, that server B is slightly faster (mean response time is smaller). The question is which test should I use to show statistical significance in difference of response times of those two servers?

• If, as you say, you have the population rather than a sample, then it is arguable whether you can or should do any significance tests at all (this has been discussed here quite a bit). – Peter Flom Jan 24 '13 at 11:18