I have a bunch of web page response time data, and I'm recording dispersion stats--both variance and quartile--on an hourly basis.
Are there good ways to roll this up into larger time scales (e.g., daily, weekly), as opposed to having to reprocess the original dataset? I understand that in the initial hourly rollup I'm losing some information, so I may have to have estimated values for the daily, weekly etc rollups, and that's fine as long as the estimate is reasonably close.
One thought that I have is that in the hourly rollups I can model the distribution of data, and then aggregate the models themselves for the daily/weekly/etc rollups. I haven't actually done the work yet of determining appropriate models for that hourly data, but conceptually this seems like a plausible direction. (Just as an aside, are there distributions that are generally accepted as being reasonable models for web page response times?)
Another idea would be to have some pre-defined bins--e.g., [0, 0.5), [0.5, 1.0), [1.0, 2.0), etc., and store counts. Of course I could have more bins for better resolution.
Are there other approaches worth exploring?
I'm not necessarily looking for theoretically perfect solutions here. Basically I'm putting together an operational dashboard for a bunch of web applications, and I need a practical way to allow operational staff to view dispersion statistics at different timescales.