2
$\begingroup$

I understand that this is a popular topic in the field of networking and telecommunication [see https://en.wikipedia.org/wiki/Event_correlation . In my problem context, I have to find correlation between logs generated in a large network, mine patterns, and generate rules to create a filtering mechanism that will identify the root cause of a series of events based on patterns from historical data.

So far the approach that I have taken involves using apriori algorithm to mine patterns of co-occurrences of events in specific time frames.

Would love to know if there are other methods currently being used or under research in this field. Would be great if someone can point me to relevant resources.

$\endgroup$
1
$\begingroup$

In case some one is working on a similar problem and lands up on this page, here is what worked for me. After trying several approaches, the method that worked best for me was to mine rules from the data set using apriori and use the 'Lift' parameter from rules as an input to a classification algorithm like random forest, that takes the attributes of two events along with the 'lift' and does a binary classification, '1' the events are correlated, '0' the events are not correlated. This system has been in production since a few months and is working really well!

$\endgroup$
  • $\begingroup$ Hi, can you share your dataset, if its not your production or your infra related data!. $\endgroup$ – iDroid Feb 13 at 17:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.