# What algorithm can I use to find correlations between events?

I am new to machine learning so I am trying to find some literature but I'm not even sure what to Google for. My data is of the following form:

User A performs Action P
User B performs Action Q
User C performs Action R
...
User C performs Action X
User A performs Action Y
User B performs Action Z
...


Where each Action has certain characteristics (date, time, client, etc). There are about 300 users and we have about 20,000 actions.

Question:

I want to find out if there is any causality/correlation between user actions. For example, "every time User E performs Action T, 2 days later User G performs Action V". But in between, there could be many other users performing many other actions, and it is possible that there is no correlation to be found. It is also possible that some users are correlated, but others are completely independent. Is this something that machine learning would be able to find for me? Is there a specific algorithm or set of algorithms that could help me?

I have been reading about Association Analysis and the Apriori algorithm but I don't think this will give me what I need, as it seems to require known, well-delimited datasets as input, whereas I just have a long stream of seemingly random user actions. Any suggestions on what to look at would be most appreciated!

• Did you try to put the event on a directed graph $G=(V,E)$, a vertex represents an action type e.g. P. There is an edge $(i,j)$ iff some user did action $i$ before $j$. it can be easily weighted with the number of user that had done such a sequence. You can also do graph per user.
– 0x90
Jan 25 '17 at 5:24

I think you've to do some data preparation before using any algorithm to find frequent items set and association rules.

In your case you need to set (and fine-tune) a max time span between interaction expected to be correlated, then you can pick a frequent user, and for each transaction he made (or a sample), you'll attach in a single record 2 days of other user transaction. Attributes should be boolean such as

UserA,transactionP |UserB,transaction Z| UserB, transaction F | [...]

• That's really helpful, thanks! So would I run the algorithm once for each user, asking it if any other user is correlated to that user, or is it possible to run it just once, and ask "show me sets of users that are correlated"?
– Matt
Jun 25 '15 at 15:33
• By following my approach you could only test a "pattern" started by a couple user+tra sition. Which users and actions are to be chosen it depends on your knowledge of possible correlation. Jun 25 '15 at 21:32
• Three suggestions. 1. Narrow down your inquiry. Not to be disrespectful, but there is no single, wondrously comprehensive statistical procedure that can tell you all of the meaningful associations out of the many, many possible ones inherent in your data. 2. Read up on time series analysis. 3. Read up on methods of distinguishing causation from mere correlation. Unfortunately, I do not have any quick fix to offer! Dec 24 '16 at 19:46

One way of exploring your data is to make a table of previous action x next action. So for each event, find the next action by the same user. You could similarly tabulate previous action X delay until next action.

Then you could explore whether the previous action influences the next one. If not, then your users are "stateless".

Another possible simplification is to ignore the userids, and ask whether the frequency of each action is the same over time or varying; and if varying whether it is cyclic or shows a trend.

The answers to these questions might show that your data have little structure. Alternatively, they might raise some new hypotheses to test.

This is an interesting question. The best approach is to look into the entire dataset and create a frequency table. For example: User A is performing Action P and Y User B is performing Action Q and Z, User C performing Action R and X So similarly, there are more users performing more actions. So, there are several approaches to deal with this dataset 1)Cluster algorithm to group similar items in different buckets 2)Market basket analysis to identify the users versus actions mapping and frequency

Without going through the entire data set it is not recommended to prescribe any particular algorithm for these kind of questions.