1
$\begingroup$

I'm not sure what to ask so forgive me if this is a bit all over the place. I've written a program to analyze the time it takes to query a data structure and generated a graph which shows on the Y axis, the time a query took and on the X axis the number of items in the structure. I didn't know what to expect at all but I'm trying interpret the results and objectively describe what I've recorded.

Probably from my description you can guess I'm a complete noob in the stats arena.

What I'm getting is something like this

random graph

Couple of questions come to mind:

How do I go about inferring anything from a graph like this?

Given the erratic Y values is there a common approach to somehow normalizing these?

It also doesn't seem like a situation where you'd just kinda draw a line of best fit and go with that...

Is that even the right thing to do? From my research so far people have mentioned standard deviation, log distribution and a host of other things but not being trained in this I don't understand when to use these things and what purpose they serve as it would apply to my case.

I could upload the real graph but I'd prefer some pointers as to how to approach this generally so that I can learn from it and have a better understanding to be able to do this next time.

Graph shown is taken from http://openclipart.org/detail/170148

Improve this question
$\endgroup$
1
  • $\begingroup$ <<I've written a program to analyze the time it takes to query <<a data structure and generated a graph which shows on the Y <<axis, the time a query took and on the X axis the number of <<items in the structure. It does not match what you have plotted, which looks more like a measure over different time of the day. Can you clarify that point ? $\endgroup$
    – Lucas Morin
    Commented Apr 9, 2013 at 11:55

2 Answers 2

Reset to default
2
$\begingroup$

heavily edited

first, I can't really fully answer your question but hopefully I can point you in the right direction. The theme here is: you are going to need to learn some stats :(.

1) You need a measure of variability. So you mentioned you measured 10 items, each one time? Do it over and over again. Figure out whether the amount of time for a query is less or more stable for each number of items. This is your "standard deviation".

Have a look at this... this is a good way to visualize the data: http://en.wikipedia.org/wiki/Violin_plot

Also Google "interquartile range" and "kernel density estimation"

2) You are going to be interested in whether the relationship is linear, or nonlinear. By this I mean: can you fit a straight line to the data, or is it better explained by a curve? In a linear world, if you query 10 items and it takes 10 nanosecond, you could predict that 11 items should take 11 nanoseconds. But this likely is not true if you query enough items. Does your curve have a "kink" in it? How many items do you need to query before you see that kink? You might be able to identify a bottleneck if someone tries to query too many items at once.

Some math: http://cstl.syr.edu/fipse/graphb/unit8/unit8.html

3) It would also be interesting to know if there are some days, or times of the day, with more variability than others. Is this the same across days? Is there a particular time of day where queries lag the most? Important to note that the standard deviation could either represent the deviation PER query, or, you could do it across days (to see if, over the 52 days you recorded data, for example, whether there are some points in the day that show greater variability in lag than others).

... this should help you identify what terms you should Google! :)

cheers, joseph

Improve this answer
$\endgroup$
3
  • $\begingroup$ I think I have read the original question differently. Although it mentions "time" and (misleadingly) shows a time series graphic, it does not appear to involve any kind of time series, nor does it involve "days." Unfortunately this appears to make most of your points less than relevant, especially #1, 4, and 5. $\endgroup$
    – whuber ♦
    Commented Apr 9, 2013 at 4:07
  • $\begingroup$ Ah,looks like you read the original question correctly as well. I'd guess he should go with a histogram / kernel density estimator. $\endgroup$
    – jdv
    Commented Apr 9, 2013 at 7:10
  • 2
    $\begingroup$ What I meant by time is that if there are 10 items I have gone through each and recorded how long (in nano seconds) it takes to access each value. So it isn't a sample as such there's a time measurement for each record. Unless that changes things significantly I think I've got a much better idea of what I need to be looking into $\endgroup$
    – zcourts
    Commented Apr 9, 2013 at 7:34
3
$\begingroup$

How do I go about inferring anything from a graph like this?

You can a get a sense of the noise level and that there is a general trend. That will suggest fitting a line or other curve, possibly a "non-parametric" curve like a spline or loess.

Given the erratic Y values is there a common approach to somehow normalizing these?

Most fitting methods are designed to handle random noise just fine. You should check that after each fit by looking at the residuals of the fit to look for non-randomness.

It also doesn't seem like a situation where you'd just kinda draw a line of best fit and go with that...

No, but it can be a useful starting point. Even if the trend is not purely linear, a line can give you an estimate of an rough trend.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.