1
$\begingroup$

I have a question concerning data that oscillates between two primary value ranges, and how one might go about determining any performance metrics or abnormalities in that data.

Example:

I have a text file of values that cluster at approximately 6000, and approximately 12000; the low value, and the high value respectively. I wish to determine some useful metric to perform on this data over a time range, to determine times when the data was "abnormal." How would one go about determining variations in this data, plus or minus a certain percentage? My current tactic is to take an average just so I can display some kind of metric, but clearly this is not the way to go for this data set.

Improve this question
$\endgroup$
3
  • $\begingroup$ What would it mean for the data to be "abnormal"? Are you interested in determining on-line if something about the underlying system has changed? $\endgroup$
    – gung - Reinstate Monica
    Commented Dec 13, 2012 at 19:09
  • $\begingroup$ Yes. For example, I'm running computer benchmarks. If the data comes out at less than "normal" that might indicate an underlying problem with the system. $\endgroup$
    – jyaworski
    Commented Dec 13, 2012 at 20:27
  • $\begingroup$ So "normal" means data consistent with some underlying probability, $p$, of being 1200, and "abnormal" means data that came from / were generated according to a different underlying probability, $p'$, is that right? Moreover, you are really only interested in if $p'<p$ (ie 'less than "normal"')? $\endgroup$
    – gung - Reinstate Monica
    Commented Dec 13, 2012 at 21:40

1 Answer 1

Reset to default
0
$\begingroup$

I would start with a graph of the behavior of the series over time. Given that there are two normal values, there are then 5 possibilities at each time point: Below the low; near the low; between low and high; near the high; above the high.

The graph should give you an idea of how much noise there is in the data at both the low and high values. You can then easily define the 5 ranges above more precisely, varying the ranges for "near" to suit your data and your needs.

As usual (always?) substantive knowledge should guide the statistics. What values are problematic, not from a statistical point of view but from a substantive one? How much measurement error is there? How much random fluctuation?

Improve this answer
$\endgroup$
2
  • $\begingroup$ So far, I like this solution. The only question I have would be how one would calculate each bin. For example, what would the boundaries be between each bin? If I use outliers to calculate that, it might influence the result. $\endgroup$
    – jyaworski
    Commented Dec 13, 2012 at 21:39
  • $\begingroup$ Use your eyes to calculate them, based on what you see in the graph. Outliers don't influence these cutoffs. $\endgroup$
    – Peter Flom
    Commented Dec 13, 2012 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.