0
$\begingroup$

I am looking at clickstream data - sequence of page visits of a user in a website. I am trying to identify anomalies in a session that could be related to bots.

To do so I am looking at a user's session and calculating the time difference between every two consecutive page impressions (views). I am calculating this time difference for all the "good-normal" users for any possible page change in the system, as a benchmark. What I see is that every time difference of every pair of pages, has a different distribution, and this is a reasonable assumption. The distribution is not normal distribution.

I would like to assess a new session - whether it follows the normal users behavior or it has an anomaly, based on the time differences I observe in the session. I had in mind to try Pearson Chi Square, but since every page change pair has a different distribution it won't work since I don't have many observations per session - I have only, let's say, 20 page impressions - meaning 19 page changes.

$\endgroup$
0
$\begingroup$

I think the easiest way is to look at the user-agent in the HTTP request. Good at catching 80-90% of bot traffic.

Look for old versions of browsers and strange things like IE6.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.