I am using Herfindahl Index metrics to measure the degree of concentration of posts by email, device_id, IP and other variables to identify potential fraud events. For example, a high degree of concentration (something above 5,000) in the degree of concentration of posts per IP would imply a possible attack.

I am measuring this metrics daily per website (the system monitors over 2,000 social media websites), but each website has a different post events count per day. So, one website may have 10 counts per day while other may have 400. Even the same website may have widely different counts per day, ranging from 10 to 1,500. Given the way Herfindahl Index is computed, this makes that the same website may have very volatile metrics day after day, and I can not build an alert system with this.

In particular, the Herfindahl Index tends to be much higher whenever the daily count of posts is low, and viceversa.

Is there a mathematical way to express the fact that for lower post counts, these indexes tend to be much higher? I remember reading about a law that expresses this fact, as it related to percentages or rates.

Beyond this, how can I build an effective alert system that accounts for the fact that higher index values will arise whenever there is a lower count of daily posts?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.