8
$\begingroup$

I have data from a load test of a web site with several thousand data points spread out over roughly 30 minutes (the values are the response time of the site in milliseconds). The values are spread out among the 30 minute range, but not at a constant rate (i.e. there may be a few milliseconds between some points, other points maybe at the same timestamp, etc).

I'd like to present this data visually and chart it, but I'm not sure of the best method for doing so - there is a good amount of variance around any sort of concept of average values or a trend line.

Is there any generally accepted best practices for methods on how to graph data of this type? I'm concerned about choosing a poor method for averaging/smoothing out the data and misrepresenting the data - such as underweighting some outlier values.

I've played around with a line chart with the timestamps on the x-axis and the average of samples in the same minute on the y-axis. I'd also like to consider graphing a moving average of the data, but I'm unsure if I should be averaging datapoints in the same N minutes or a window of the last N points.

I'd like to make sure that whatever choice I make would appear to be a rigorous representation of the data and not too amateur-ish.

Update: below is a sample of what I have produced so far, each point on the chart is taken as the mean/median of all of the samples within the same minute (i.e. within 11:12:00.000 and 11:12:59.999). I included the number of samples per minute as a bar chart in the second half of the image to be able to show if any single points in the line chart look as outliers due to a small amount of samples, although aesthetically speaking I think the bar chart takes up way too much real estate for the amount of information it gives.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
6
$\begingroup$

I suggest adding an example or two of what you are presently doing so we can better see what you are dealing with.

What you are concerned with is an important issue: how do you convey the "overall" pattern in the time series data while also not misleading viewers by showing just average values? One way I have dealt with this situation is plotting an average or median line along with surrounding quantile bands. For example,

enter image description here

Here, the time series data are from a bootstrap-based simulation so there are hundreds of values associated with each time point. The actual data are plotted in the black line with colored bands showing the variability of values from the simulation. This particular plot is maybe not the best example to show, but you can see that some points have much more variability than others, and you can also assess how the variability is skewed above/below the actual values depending on the position in the series.

UPDATE: Given your update here are some additional questions and thoughts... What decisions, if any, are made from this visualization? For example, are you looking for specific points in time where there is very slow response time, perhaps above a specific threshold? If so, it may be better to simply plot all of the points as a scatter plot, and then also plot a time series line showing the average value, as well as some lines delineating the bounds you are concerned about. This recommendation is not appropriate if you have numerous observations at some time points (too much clutter), or if your time measurement is not sufficiently coarse (in which case you can bin response data into minute-wide time of day intervals). But the visualization recommendation will certainly be affected by what decision(s) will be supported with it. In my example, I was looking at such plots side by side, one from one simulation and the other from another simulation (each simulation using different parameters) so I could assess the variability of the underlying model due to sampling error.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks, I like this idea a lot. I also updated my main question with an example of what I've done so far as requested. Is it fair of me to average the data points into minute buckets, when each bucket may not have the same amount of samples in it? $\endgroup$
    – matt b
    Commented May 25, 2011 at 13:26
  • 1
    $\begingroup$ Another possibility would be to plot density strips for each time period as opposed to discrete colors for quantiles, see this article in the American Statistician for examples dx.doi.org/10.1198/000313008X370843 . The graph posted seems pretty informative as is though (+1) $\endgroup$
    – Andy W
    Commented May 25, 2011 at 16:36
  • $\begingroup$ @Andy W, do you happen to have an example image of that method? It seems like I can only read the abstract of that article without a login. $\endgroup$
    – matt b
    Commented May 26, 2011 at 14:15
  • 1
    $\begingroup$ @matt b, I just found an ungated version of the paper. Also check out the R package denstrip. In retrospect though I think the suggestion may be overkill, you will only have slight gains over just plotting the quantiles of interest and it will be much more computationally expensive (hence reducing its utility as an exploratory tool). $\endgroup$
    – Andy W
    Commented May 26, 2011 at 14:27
3
$\begingroup$

Have you considered a scatterplot of the data themselves? That's an approach I really like. It lets the viewer make their own conclusions about the presence and significance of trends, and it doesn't conceal variability or outliers. Alpha-blending the points will help if you have serious overplotting (which it sounds like you might). You can also overlay whatever trend you like and take comfort in knowing that the data are still present to speak for themselves.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Matt, just to clarify on the "alpha-blending" comment: I assume you mean setting the alpha channel of the graphics to allow for some level of transparency in displayed graphical elements? $\endgroup$
    – Josh Hemann
    Commented May 25, 2011 at 16:22
  • $\begingroup$ I did attempt a scatterplot, but I found that I had such tight clustering of the datapoints for each x-value that it's a bit hard to tell, personally, just how many values appear at certain levels - see i.imgur.com/FsNC1.png for example. I'm nervous that a chart like this doesn't make it obvious just how many points appear in the cluster around the y=0 line $\endgroup$
    – matt b
    Commented May 25, 2011 at 16:32
  • $\begingroup$ I think this may be a good approach for visualizing outliers, I'm not so sure it is a great idea for the whole distribution though. Also as a note in regards to your linked question, Tufte has an example in his Visual Display of Quantitative Information where plotting the lines between the temporal dimension (similar to your chart 1 on the linked question) shows an obvious periodic pattern, whereas it is much more difficult to discern with just the scatterplot without the lines. $\endgroup$
    – Andy W
    Commented May 25, 2011 at 16:42
  • $\begingroup$ @Josh Right - as in this image $\endgroup$
    – Matt Parker
    Commented May 25, 2011 at 17:35
  • 1
    $\begingroup$ @matt That's a classic case for alpha-blending (see the image in the comment above). It requires some experimentation with the level of transparency, but you'll likely be able to find something that works. It still won't be perfectly clear how many points are at each spot on the axis, but it gives the viewer a sense of density that lines don't. $\endgroup$
    – Matt Parker
    Commented May 25, 2011 at 17:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.