Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have some data on the number of times each of my machines turned off (due to an error) in a particular time period. There are about 6 different classes of machines being used to construct a total population of 50 machines. I wanted to analyze the stability of the 6 classes of machine relative to each other.

An acquaintance suggested that I could do some heavy-hitter analysis during a brief chat to determine if some machines are shutting down more than others. Can someone tell me how to systematically perform this analysis or if there is a formal name for this kind of analysis?

share|improve this question
I've never heard of this before, but perhaps this presentation will be of some help. "Frequent item analysis" seems to be the formal name. – Matt Parker Jun 11 '11 at 19:10

Upon digging deeper into this question, I figured out that this is nothing but extracting the heavy hitters (or the tail or the outliers) from the CCDF plot (which is 1-ECDF). If you are using R, one can do this in the following way:

Y = read.table(...)
Y.ecdf = ecdf(Y$V1)
curve((1-Y.ecdf(x)), n = 10000, 
       from = 0, to = 2600, ylab = "Pr(X > x)", 
       xlab = "x", col="blue", lty=1, lwd=2)

For example, the points on the extreme right (say from 80-100 but of course this is purely domain-specific) are the heavy-hitters or the tail of the curve.

enter image description here

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.